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Abstract 


Frames  are  used  in  many  signal  processing  applications.  We  consider  the  problem  of 
constructing  every  frame  whose  frame  operator  has  a  given  spectrum  and  whose  vectors 
have  prescribed  lengths.  For  a  given  spectrum  and  set  of  lengths,  we  know  when  such  a 
frame  exists  by  the  Schur-Hom  Theorem;  it  exists  if  and  only  if  its  spectrum  majorizes 
its  squared  lengths.  We  provide  a  more  constructive  proof  of  Horn’s  original  result.  This 
proof  is  based  on  a  new  method  for  constructing  any  and  all  frames  whose  frame  operator 
has  a  prescribed  spectrum  and  whose  vectors  have  prescribed  lengths.  Constructing  all 
such  frames  requires  one  choose  eigensteps — a  sequence  of  interlacing  spectra — which 
transform  the  trivial  spectrum  into  the  desired  one.  We  give  a  complete  characterization  of 
the  convex  set  of  all  eigensteps.  Taken  together,  these  results  permit  us,  for  the  first  time, 
to  explicitly  parametrize  the  set  of  all  frames  whose  frame  operator  has  a  given  spectrum 
and  whose  elements  have  a  given  set  of  lengths.  Moreover,  we  generalize  this  theory  to 
the  problem  of  constructing  optimal  frame  completions.  That  is,  given  a  preexisting  set 
of  measurements,  we  add  new  measurements  so  that  the  final  frame  operator  has  a  given 
spectrum  and  whose  added  vectors  have  prescribed  lengths.  We  introduce  a  new  matrix 
notation  for  representing  the  final  spectrum  with  respect  to  the  initial  spectrum  and  prove 
that  existence  of  such  a  frame  relies  upon  a  majorization  constraint  involving  the  final 
spectrum  and  the  frame’s  matrix  representation.  In  a  special  case,  we  provide  a  formula 
for  constructing  the  optimal  frame  completion  with  respect  to  fusion  metrics  such  as  the 
mean  square  error  (MSE)  and  frame  potential  (FP).  Such  fusion  metrics  provide  a  means 
of  evaluating  the  efficacy  of  reconstructing  signals  which  have  been  distorted  by  noise. 
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PARAMETRIZING  FINITE  FRAMES  AND  OPTIMAL  FRAME  COMPLETIONS 


I.  Introduction 

Frames,  or  redundant  sets  of  vectors  in  a  Hilbert  space,  have  been  used  in  many  signal 
processing  applications  and  are  the  subject  of  the  research  discussed  herein.  We  begin  with 
a  few  important  definitions  regarding  frames.  A  sequence  of  vectors  F  =  {fn}N=1  is  a  frame 
in  an  M-dimensional  (real  or  complex)  Hilbert  space  Mm  if  there  exist  0  <  A  <  B  <  oo 
such  that 

N 

A\\f\\2<J]\(f,fn)\2<B\\f\\2 

72=1 

for  all  /  in  Mm.  It  is  straightforward  to  show  that  F  is  a  frame  if  and  only  if  the  F  spans 
Hm,  which  necessitates  M  <  N.  Letting  K  be  either  the  real  or  complex  field,  the  synthesis 
operator  of  a  sequence  of  vectors  {/„|^=1  is  F  :  KN  — »  iM,  Fg  :=  Xn=\  g(n)fn-  The 
corresponding  frame  operator  FF*  :  WM  — >  EIM  is  given  by: 

N 

FF*f  =  J](f,fn)fn,  (1.1) 

72=1 

where  *  denotes  the  Hermitian  transpose  of  F.  Viewing  as  KM,  F  is  the  M  x  N  matrix 
whose  columns  are  given  by  the  /„’ s.  The  constants  A  and  B  are  called  the  lower  and  upper 
frame  bounds  for  F,  respectively.  Since  is  finite-dimensional,  then  the  frame  bounds 
of  F  are  the  least  and  greatest  eigenvalues  of  the  frame  operator.  That  is,  |  is  the  condition 
number  of  FF*  and  F  is  a  frame  if  and  only  if  FF*  is  invertible. 

For  a  given  frame  F,  the  canonical  ducd frame  F  =  {f„}^=vf„  :=  ( FF*)~lf, ,  provides 
a  means  of  expressing  any  /  e  IHIM  as  a  linear  combination  of  frame  vectors,  that  is: 

N 

f  =  FF*f  =  YJ(fJn)fn.  (1.2) 

72=1 
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The  canonical  dual  frame  of  F  satisfies  FF*  =  I.  Unlike  bases,  frames  provide  a  means 
of  representing  any  /  e  in  terms  of  overcomplete  sets  of  vectors.  The  fact  that  frames 
may  be  redundant  make  them  useful  in  many  signal  processing  applications  where  data  loss 
and  degradation  are  concerns  [35,  36]. 

In  particular,  frames  have  found  a  natural  application  to  source  coding  and  robust 
transmission.  Frames  have  been  shown  to  provide  numerically  stable  reconstruction  and 
resilience  to  additive  noise  [27].  While  not  always  fully  realistic,  additive  noise  can  be  used 
to  model  channel  noise  and/or  quantization  errors  which  are  introduced  in  analog-to-digital 
conversion.  In  addition  to  noise,  erasures  of  frame  coefficients  can  also  be  problematic 
when  transmitting  a  signal  [13,  26,  31].  The  redundancy  of  frames  allows  signals  in 
which  data  has  been  lost  to  be  reconstructed  in  a  feasible  manner.  Such  applications  have 
motived  current  research  efforts  to  find  ways  of  constructing  frames  with  certain  desirable 
properties. 

The  focus  of  our  research  is  on  constructing  finite  frames  for  a  given  spectrum  and  set 
of  lengths.  This  is  a  generalization  of  a  problem  that  the  field  has  long  been  interested  in: 
constructing  unit  norm  tight  frames  (UNTFs).  A  tight  frame  is  one  for  which  A  =  B.  and 
a  UNTF  is  a  tight  frame  with  the  additional  condition  that  ||/„||  =  1  for  all  n  =  l, ...  ,N. 
In  this  dissertation,  we  show  how  to  explicitly  construct  all  UNTFs  and  moreover,  how  to 
construct  any  frame  for  a  given  arbitrary  spectrum  and  set  of  lengths.  This  theory  has  also 
been  generalized  to  the  problem  of  constructing  frame  completions',  given  a  preexisting  set 
of  measurements,  we  add  new  measurements  so  that  the  final  frame  operator  has  a  given 
spectrum  and  whose  added  vectors  have  prescribed  lengths.  More  formally,  given  an  initial 
sequence  of  vectors  FN  =  {fn}^=l  in  WM  whose  frame  operator  has  spectrum  [arm}£f=1,  we 
wish  to  complete  the  frame  by  adding  P  additional  measurements  in  order  to  construct 
F  =  {fn\^f\  such  that  \\fN+p\\2  =  pN+p  for  all  p  =  1, . . .  ,P,  and  such  that  the  new  frame 
operator  of  F  =  {fn)"fp  has  spectrum  \f  „\"=v 
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Furthermore,  we  provide  some  results  on  optimal  frame  completions.  For  real  world 
applications,  we  often  want  to  choose  a  frame  that  is  optimal  with  respect  to  certain  criteria. 
For  example,  in  cases  where  part  of  a  signal  being  transmitted  is  distorted  by  noise  or  lost 
due  to  network  errors,  an  optimal  frame  would  be  one  that  minimizes  the  reconstruction 
error.  Goyal,  Kovacevic,  and  Kelner  [26]  used  the  mean  square  error  (MSE)  as  a  means  of 
evaluating  the  quality  of  reconstruction  in  the  former  case.  The  MSE  is  given  by: 

m  . 

MSE  =  cr2Tr[(FF*)-1]  =  cr2  Y  —  (1.3) 

Am 

where  ]  Am  }^= ,  are  the  eigenvalues  of  FF*  and  cr2  is  the  variance  of  the  zero-mean, 
independent,  identically  distributed  added  noise  e  =  [ei  e2  . . .  ey]1.  Since  calculating  the 
MSE  involves  inverting  the  frame  operator,  which  can  be  difficult,  the  frame  potential  may 
also  be  used  as  an  alternative  notion  of  the  quality  of  a  frame.  The  frame  potential  (FP)  of 
a  sequence  {/„}^=|  in  KN  is  given  by: 

M 

FP ({/„}"  i)  =  Tr[(EF*)2]  =  d2  (1.4) 

m-\ 

where  again,  {Am}^=1  are  the  eigenvalues  of  FF*.  It  has  already  been  shown  that  a  unit  norm 
frame  minimizes  the  MSE  and  FP  if  and  only  if  it  is  tight  [5,  26].  However,  constructing  a 
UNTF  for  a  given  application  may  not  always  be  achievable  if,  given  an  initial  frame,  one  is 
restricted  to  adding  only  a  finite  number  of  new  measurements.  In  such  cases,  determining 
optimality  of  a  frame  is  not  as  straightforward.  We  provide  an  algorithm  for  constructing 
the  optimal  frame  completion  with  respect  to  fusion  metrics  such  as  the  MSE  and  FP. 

1.1  Major  contributions 

In  this  section,  we  briefly  summarize  the  major  contributions  that  have  arisen  from 
our  doctoral  studies  [8,  24,  25].  Our  first  major  contribution  (Section  3.1)  is  Theorem  2 
which  characterizes  and  proves  the  existence  of  sequences  of  vectors  that  generate  a  given 
sequence  of  outer  eigensteps.  In  particular,  for  any  {Amj^=l  and  {pn}„=l,  every  sequence  of 
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vectors  {/„}^=1  in  HM  whose  frame  operator  FF*  has  spectrum  {Am}^=]  and  which  satisfies 
||/„||2  =  jin  for  all  n  can  be  constructed  by  the  two-step  process  given  in  Theorem  2,  and 
conversely,  any  F  constructed  by  this  process  has  {/l,„|ff=l  as  the  spectrum  of  FF*  and 
\\fn\\2  =  Hn  for  nil  n.  However,  Theorem  2  alone  is  not  an  easily  implementable  algorithm: 
Step  A  requires  one  to  choose  a  sequence  of  interlacing  spectra  which  transform  the  trivial 
spectrum  into  the  desired  one;  Step  B  requires  one  to  compute  orthonormal  eigenbases  for 
each  Fn.  These  steps  are  addressed  in  Chapters  4  and  3,  respectively. 

Our  second  major  contribution  (Section  3.2)  is  to  provide  a  more  explicit  algorithm 
for  Step  B  of  Theorem  2.  Theorem  7  provides  a  more  explicit  iterative  algorithm  for 
constructing  every  sequence  of  vectors  F  =  {fn}%=l  in  whose  frame  operator  FF*  has 
spectrum  {Am}^=1  and  which  satisfies  \\f,\\2  =  pn  for  all  n.  The  proof  of  Theorem  7  involves 
Lagrange  interpolating  polynomials  and  a  few  additional  polynomial  identities  which  set  it 
apart  from  prior  theory.  Of  particular  interest  is  the  construction  of  UNTFs.  While  there  are 
a  few  algorithms  which  can  construct  UNTFs,  little  up  to  this  point  was  known  about  the 
manifold  of  all  UNTFs.  Two  common  ways  of  constructing  UNTFs  are  by  (i)  truncating 
a  discrete  Fourier  transform  basis  [26]  and  (ii)  using  an  iterative  method  called  Spectral 
Tetris  [12].  Our  results  go  far  beyond  the  existing  theory  in  that  they  not  only  show  how 
to  construct  all  UNTFs,  but  they  also  show  how  to  explicitly  construct  every  frame  whose 
frame  operator  has  a  given  arbitrary  spectrum  and  whose  vectors  are  of  given  arbitrary 
lengths. 

The  third  major  contribution  (Section  4.2)  has  been  to  provide  an  explicit  algorithm, 
called  Top  Kill ,  for  constructing  a  sequence  of  inner  eigensteps  {{/l„;m}^=1}^=  0  which  satisfy 
Definition  12.  Given  in  Theorem  16,  this  algorithm  refines  Step  A  of  Theorem  2  by  giving 
an  explicit  construction  of  a  feasible  set  of  inner  eigensteps  whenever  {d„}^=1  majorizes 
{pn\tr  The  Schur-Horn  Theorem  gives  that  this  set  is  nonempty  if  and  only  if  {An}^=l 
majorizes  {pn\Nn=v  Horn  and  Johnson  [33]  already  proved  that  such  a  sequence  exists, 
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however,  their  proof  is  not  constructive.  The  Top  Kill  algorithm  presented  not  only  provides 
an  alternative  proof  of  Horn  and  Johnson’s  result,  but  it  provides  a  method  to  explicitly 
construct  a  sequence  of  eigensteps  as  required  by  Step  A  of  Theorem  2. 

Our  fourth  contribution  (Section  4.3)  as  stated  in  Theorem  17  gives  a  complete 
characterization  of  the  set  of  all  eigensteps  for  a  given  {d„}^=1  and  {pn}^=v  Unlike  Top 
Kill  which  gives  one  strategy  for  constructing  one  sequence  of  eigensteps,  Theorem  17 
provides  an  algorithm  for  constructing  all  such  sequences.  We  further  show  in  Chapter  4 
that  the  set  of  all  such  sequences  is  convex.  Taken  together,  these  results  permit  us,  for  the 
first  time,  to  explicitly  parametrize  the  set  of  all  frames  whose  frame  operator  has  a  given 
spectrum  and  whose  elements  have  a  given  set  of  lengths. 

The  next  two  major  contributions  are  results  of  the  frame  completion  problem;  that 
is,  given  a  sequence  of  vectors  FN  =  {f„\*=v  we  wish  to  complete  the  frame  by  adding 
P  additional  vectors  {/n+p}pp={  such  that  ||/,'V+/J| |2  =  i_iN+p  for  all  p  =  1  and  such 

that  the  new  frame  operator  of  F  =  {/„}^2jp  has  spectrum  \  Am  }^= , .  In  Chapter  5,  we 
determine  what  it  means  for  a  sequence  {Am}^=l  to  be  (cr,/r)-constructible  (Theorem  39) 
and  show  that  constructibility  relies  upon  a  majorization  constraint  involving  {Am}ffI=i  and 
the  frame’s  spectral  partition  matrix.  The  second  major  contribution  in  Chapter  5  is  a 
new  algorithm  called  Chop  Kill  (Theorem  40)  which  is  a  generalization  of  the  Top  Kill 
algorithm  in  Chapter  4.  This  algorithm  enables  us  to  explicitly  construct  a  valid  sequence  of 
continued  outer  eigensteps  whenever  {DS(A)y}4f=1  majorizes  {p-N+P}p=l,  be.,  for  any  ( a,p )- 
constructible  sequence.  Having  a  valid  sequence  of  eigensteps,  we  then  return  to  Step  B 
of  the  two-step  process  and  use  the  algorithm  given  in  Theorem  7  in  order  to  explicitly 
construct  the  added  frame  vectors  {/v+P}P=1- 

Finally,  our  last  major  contribution  (Section  6.2)  as  stated  in  Theorem  48,  is  a  closed- 
form  solution  for  the  optimal  (a,yu)-constructible  sequence  in  the  special  case  that  \pN+f)}l’i=l 
are  all  of  equal  lengths.  To  prove  this  result,  we  measure  optimality  with  respect  to 
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majorization,  and  show  that  the  optimal  (o\p)-constructiblc  sequence  is  the  the  one  which 
is  majorized  by  all  other  sequences  which  can  be  built  from  {am}^=1  and  {pN+p\Fp=v  i.e.,  all 
other  (a, //)-constructible  sequences. 

The  following  is  a  list  of  current  publications  that  have  arisen  from  my  PhD  studies: 

Journal  Articles 

1.  Cahill,  Jameson,  Matthew  Fickus,  Dustin  G.  Mixon,  Miriam  J.  Poteet  and  Nate 
Strawn.  “Constructing  finite  frames  of  a  given  spectrum  and  set  of  lengths,”  to  appear 
in  Applied  and  Computational  Harmonic  Analysis,  22  pages  (2012). 

2.  Fickus,  Matthew,  Dustin  G.  Mixon,  Miriam  J.  Poteet  and  Nate  Strawn.  “Constructing 
all  self-adjoint  matrices  with  prescribed  spectrum  and  diagonal,”  submitted  to: 
Advances  in  Computational  Mathematics,  20  pages. 

Book  Chapters 

1.  Fickus,  Matthew,  Dustin  G.  Mixon  and  Miriam  J.  Poteet.  “Constructing  all  finite 
frames  with  a  given  spectrum,”  in  Finite  Frames:  Theory  and  Applications,  P.  G. 
Casazza  and  G.  Kutyniok  eds.,  Birkhauser,  pp.  55-107  (2012). 

Conference  Proceedings 

1.  Fickus,  Matthew,  Dustin  G.  Mixon  and  Miriam  J.  Poteet.  “Frame  completions  for 
optimally  robust  reconstruction,”  Proceedings  ofSPIE,  8138:  81380Q/1-8  (2011). 

1.2  Outline 

In  Chapter  2,  we  review  the  relevant  background  literature  of  frames,  majorization, 
and  interlacing.  In  Chapter  3,  we  present  one  of  our  main  result  (Theorem  2)  which 
characterizes  and  proves  the  existence  of  sequences  of  vectors  that  generate  a  given 
sequence  of  outer  eigensteps.  We  then  present  a  second  major  result  (Theorem  7) 
which  provides  a  more  explicit  iterative  algorithm  for  constructing  every  sequence  of 
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vectors  F  =  {/„}^=1  in  H,v/  whose  frame  operator  FF  has  spectrum  {/t,„}^=1  and  which 
satisfies  ||/„||2  =  n„  for  all  n.  Chapter  4  contains  the  Top  Kill  algorithm  and  a  result 
(Theorem  17)  which  gives  a  complete  characterization  of  the  set  of  all  eigensteps,  as 
needed  by  Step  A  of  Theorem  2.  Finally,  we  discuss  frame  completions  in  Chapters  5 
and  6.  Specifically,  Chapter  5  contains  the  Chop  Kill  algorithm  and  a  result  (Theorem  39) 
which  determines  when  a  sequence  is  (cr,)u)-constructible.  Chapter  6  contains  our  final 
major  result  (Theorem  48)  which  is  an  explicit  formula  for  the  optimal  (cr,yu)-constructible 
sequence  in  the  special  case  that  {jJLN+p}pp=x  are  all  of  equal  lengths. 
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II.  Literature  Review 


In  this  chapter,  we  review  the  literature  that  is  relevant  to  our  research.  In  Section  2.1, 
we  discuss  the  relevant  frame  theory  literature.  Simply  stated,  frames  are  redundant  sets  of 
vectors  in  a  Hilbert  space.  In  contrast  to  unique  representation  of  vectors  with  respect  to  a 
given  basis,  frames  provide  a  means  of  representing  any  /  in  WM  in  terms  of  overcomplete 
sets  of  vectors.  Frames  have  many  useful  applications  including,  but  not  limited  to,  their 
resiliency  to  additive  noise  and  quantization  [27],  and  erasures  [26]. 

In  Section  2.2,  we  discuss  literature  relevant  to  majorization.  Namely,  given  two 
nonnegative  nonincreasing  sequences  [d„}^=1  and  {Hn)„=v  we  say  that  {d„}^=1  majorizes 
U/«C=r  denoted  {d„}^=1  >  \/Jn}^=r  if 

n  n 

V/7  =  1, 

n'= 1  n'= 1 

N  N 

y  ^  =  y  ^  fin' . 

n'= 1  «'= 1 

Hardy  et  al.  discussed  many  of  the  main  ideas  of  majorization  in  their  1934  book 
Inequalities  [29].  From  that  point  forward,  the  concepts  of  majorization  have  been  applied 
to  a  broad  spectrum  of  fields  from  statistical  mechanics  to  graph  theory  [2]. 

Also  in  Section  2.2,  we  discuss  the  background  literature  of  interlacing.  We  say  a 
nonnegative  nonincreasing  sequence  [ym[^=|  interlaces  on  another  such  sequence  {p,„}^=v 
denoted  [ySm("=l  E  {y,„(^=l,  provided 

Pm  <  7m  ^  Pm- i  ^  7m- i  <  ■  •  •  <  Pi<7i<  P\  <  7\- 

Under  the  convention  yM+i  :=  0,  we  have  that  \pm };jf= ,  E  [ym};^=l  if  and  only  if  ym+i  <  pm  < 
7,n  for  all  m  =  1, . . . ,  M.  In  the  context  of  our  research,  interlacing  is  applied  to  the  spectra 
of  partial  sums  of  the  frame  operator,  and  is  also  useful  in  that  it  provides  an  alternative 
way  of  viewing  majorization  [33]. 
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2.1  Frames 


Frame  theory  was  first  introduced  by  DufRn  and  Schaeffer  in  1952  in  their  paper 
on  nonharmonic  Fourier  series  [20].  The  fact  that  frames  may  be  redundant  makes 
them  useful  in  many  signal  processing  applications  where  data  loss  and  degradation  are 
a  concern  [35,  36].  In  particular,  quantization  error  is  introduced  in  analog-to-digital 
conversion,  that  is,  when  a  signal  that  is  varying  continuously  is  restricted  to  a  set  of  discrete 
values.  There,  round-off  and  truncation  errors  may  distort  the  signal.  While  not  always 
realistic,  additive  noise  can  be  used  to  model  channel  noise  and/or  quantization  errors. 
Goyal  et  al.  found  that  consistent  reconstruction  methods  yielded  smaller  reconstruction 
errors  than  that  of  classical  methods  and  that  UNTFs  are  optimally  robust  with  respect  to 
additive  noise  [26,  27]. 

Another  application  of  frames  is  that  of  data  transmission.  For  example,  when  data 
is  transmitted  over  a  communications  network  such  as  the  internet,  it  is  possible  that  the 
information  being  transmitted  can  be  distorted  or  lost  due  to  network  errors.  When  modeled 
in  terms  of  frames,  this  corresponds  to  erasures ,  namely  having  some  of  the  transform 
coefficients  are  lost  during  the  transmission.  Erasures  of  frame  coefficients  have  been 
well  studied  [13,  26,  31].  In  particular,  it  has  been  shown  that  UNTFs  are  useful  for  data 
transmission  which  is  robust  against  erasures  [13].  One  useful  class  of  UNTFs  in  this 
setting  are  harmonic  tight  frames  (HTFs)  which  are  constructed  from  a  discrete  Fourier 
transform  basis  [13]. 

In  light  of  these  applications,  we  often  want  to  choose  a  frame  that  is  optimal  with 
respect  to  certain  criteria.  The  authors  of  [5,  26,  27]  have  already  explored  metrics  such 
as  the  MSE  (1.3)  and  FP  (1.4)  as  a  means  of  quantifying  reconstruction  errors  in  the 
applications  mentioned  above,  and  have  shown  that  unit  norm  frames  minimize  these 
quantities  if  and  only  if  they  are  tight  [5,  26].  In  particular,  for  an  M  x  N  UNTF  the 
minimum  value  of  the  FP  is  precisely  jj-  [5].  In  regards  to  channel  noise  and  quantization 
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errors,  consistent  reconstruction  methods  were  shown  to  be  optimal,  with  the  MSE  of  the 
reconstruction  being  equal  to  where  cr 2  is  the  variance  of  the  added  noise  [27] .  UNTFs 

are  also  known  as  Welch-Bound-Equality  sequences  since  they  give  equality  to  the  Welch 
bound ,  that  is  Tr[(FF*)2]  =  ^  [47,  48].  These  sequences  are  shown  to  be  the  optimal 
signature  sequences  in  CDMA  systems  [46]. 

When  considering  frames  for  applications  such  as  those  mentioned  above,  we  often 
wish  to  retain  control  over  the  spectrum  of  FF*  and  the  lengths  of  the  frame  vectors  [24] . 
Of  particular  interest  is  the  construction  of  UNTFs.  It  is  known  that  the  manifold  of  all  real 
MxN  UNTFs,  modulo  rotations,  has  dimension  (N—M—l)(M—l)  when  N  >  M  [22].  When 
M  =  N  +  1,  it  is  shown  in  [26],  that  such  tight  frames  are  essentially  unique.  Additionally, 
it  is  shown  in  [43]  that  explicit  local  parameterizations  of  this  manifold  can  be  constructed. 
Since  the  columns  of  F  must  have  unit  norm  and  the  rows  must  be  orthogonal,  constructing 
a  UNTF  is  not  an  easy  task  as  it  involves  solving  a  large  system  of  quadratic  equations.  In 
fact,  only  a  few  algorithms  exist  which  can  construct  examples  of  such  frames,  and  even 
then,  they  only  construct  a  finite  subset  of  this  continuous  manifold.  Two  common  ways  of 
constructing  UNTFs  are  HTFs  [26]  and  iteratively  by  a  method  called  Spectral  Tetris  [12]. 
Tight  frames  can  also  be  constructed  via  a  method  based  on  alternating  projections  as 
discussed  in  [45]  and  by  using  Householder  transformations  [23]. 

Algorithms  constructing  UNTFs  have  also  been  proposed  to  address  the  Paulsen 
problem :  How  close  is  a  frame  which  is  almost  tight  and  almost  unit  norm  to  some  UNTF? 
This  question  arises  in  applications  when  given  an  initial  frame,  it  may  be  more  desirable 
to  transform  the  given  frame  into  a  UNTF  which,  as  described  above,  is  optimal  for  a 
variety  of  reasons.  It  is  shown  in  [1 1]  that  for  a  given  unit  norm  frame,  a  nearby  UNTF  can 
iteratively  be  found  by  a  method  of  gradient  descent  of  the  frame  potential.  Additionally, 
Bodmann  and  Casazza  [6]  have  shown  that  equal-norm  Parseval  frames  (frames  for  which 
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A  =  B  =  1)  can  be  constructed  via  a  method  based  on  a  system  of  ordinary  differential 
equations. 

Tight  frames  can  also  be  constructed  from  an  existing  set  of  vectors.  That  is,  given  an 
initial  sequence  of  vectors,  additional  vectors  with  prescribed  norms  are  added  in  order  to 
construct  a  tight  frame.  The  necessary  and  sufficient  conditions  for  the  existence  of  such 
frames  can  be  found  in  [39].  In  [23],  the  problem  of  tight  frame  completions  is  considered 
as  well  as  finding  a  lower  bound  for  the  number  of  vectors  to  add  in  order  to  construct  a 
tight  frame.  However,  as  the  authors  of  [23]  point  out,  when  the  added  vectors  are  required 
to  be  unit  norm,  the  lower  bound  they  propose  is  not  a  true  lower  bound.  Massey  and 
Ruiz  [39]  considered  this  problem  as  well,  and  calculate  the  minimum  number  of  vectors 
to  add  regardless  of  whether  they  are  unit  norm  or  not.  Additionally,  they  provide  an 
algorithm  for  constructing  a  tight  frame  completion  from  an  initial  sequence  of  vectors. 

For  real  world  applications,  it  may  be  desirable  to  complete  the  frame  in  such  a  way 
that  it  is  optimal  with  respect  to  certain  criteria.  For  example,  in  cases  where  part  of  a 
signal  being  transmitted  is  distorted  by  noise  or  lost  due  to  network  errors,  an  optimal 
frame  would  be  one  that  minimizes  the  reconstruction  error.  Constructing  a  UNTF  for  a 
given  application  may  not  always  be  achievable  if  one  is  restricted  to  adding  only  a  finite 
number  of  new  vectors  in  order  to  complete  the  frame.  Moreover,  in  this  setting,  it  is  not 
straightforward  to  optimally  complete  the  frame.  Massey,  Ruiz,  and  Stojanoff  show  in  [40] 
and  [41]  that  determining  optimality  of  a  frame  completion  does  not  have  to  be  limited  to 
the  MSE  and  the  FP,  but  rather  can  be  measured  with  respect  to  majorization.  In  short,  they 
show  that  optimal  frame  completions  are  minimizers  of  a  family  of  convex  functionals, 
that  includes,  but  is  not  limited  to  the  MSE  and  FP.  Algorithms  for  solving  for  the  optimal 
frame  completion  can  be  found  in  [40]  and  [41];  however,  as  the  authors  note  in  [41],  the 
algorithm’s  performance  in  cases  when  a  larger  number  of  vectors  is  added  or  when  the 
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dimension  of  the  space  is  large  is  contingent  upon  a  conjecture  which  they  have  yet  to 
prove. 

As  we  will  show  in  the  chapters  that  follow,  for  the  first  time,  we  have  been  able  to  go 
beyond  the  existing  theory  of  constructing  UNTFs  by  showing  how  to  explicitly  construct 
every  frame  belonging  to  the  manifold  of  all  real  UNTFs.  Moreover,  we  provide  a  partial 
solution  to  the  frame  completion  problem  we  posed  in  [24]  and  present  an  algorithm  for 
finding  the  optimal  frame  completion  in  a  special  case  which  can  be  implemented  in  a  finite 
number  of  iterations.  Being  able  to  explicitly  construct  every  frame  whose  frame  operator 
has  a  given  arbitrary  spectrum  and  whose  vectors  are  of  given  arbitrary  lengths  requires 
existing  theory  of  majorization  and  the  Schur-Hom  Theorem  which  are  discussed  in  the 
following  section.  Relevant  interlacing  literature  is  also  discussed. 

2.2  Majorization  and  interlacing 

Majorization  arises  in  a  variety  of  contexts  including  combinatorial  analysis,  matrix 
theory,  numerical  analysis,  and  statistics  just  to  name  a  few  [2,  29].  Here,  we  are  interested 
in  how  the  majorization  inequalities  apply  to  matrix  theory.  Specifically,  the  concept  of 
majorization  was  used  by  Schur  in  [42]  where  he  showed  that  the  spectrum  of  a  self- 
adjoint  positive  semidefinite  matrix  necessarily  majorizes  its  diagonal  entries.  Some 
time  later,  Horn  [32]  proved  the  converse:  if  { An  then  there  exists  a  self- 

adjoint  matrix  that  has  [T„}^=1  as  its  spectrum  and  {ju„}^=l  as  its  diagonal  [32].  Combining 
these  two  results  gives  the  Schur-Hom  Theorem  which  states  that  there  exists  a  positive 
semidefinite  self-adjoint  matrix  with  spectrum  |  An  }^=  t  and  diagonal  entries  {jj„}n=1  if  and 
only  if  >  {^n)n=v  Alternative  proofs  of  this  theorem  are  given  in  [37]. 

Existence  of  frames  with  predefined  lengths  have  been  studied  in  [10,  21].  In  regards 
to  tight  frames,  Casazza  et  al.  gave  necessary  conditions  upon  a  set  of  lengths  ,  to 
prove  existence  of  tight  frames  for  which  ||/J,||  =  /jn  for  all  n.  Moreover,  it  is  shown  that 
for  any  set  of  lengths  {n„}„=l  which  satisfy  the  fundamental  inequality ,  max„=1  jv(/u)  < 
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there  exists  a  tight  frame  for  which  |[/„j  =  //„  for  all  n.  Viswanath  and 
Anantharam  [46]  independently  discovered  the  fundamental  inequality  when  constructing 
optimal  signature  sequences  for  CDMA  systems.  The  connection  between  constructing 
optimal  CDMA  signature  sequences  and  the  Schur-Horn  Theorem  is  discussed  in  [44],  and 
finite-step  algorithms  for  constructing  such  sequences  are  given. 

In  context  of  our  research,  by  applying  the  Schur-Horn  Theorem  to  the  Gram  matrix 
F*F,  whose  diagonal  entries  are  given  by  {||/„||2}^=1  and  whose  spectrum  is  ]d„}^=1,  we  are 
able  to  determine  when  a  frame  with  prescribed  spectrum  and  lengths  exists.  Here  it  is 
important  to  note  that  the  spectrum  of  the  Gram  matrix  F*F  is  a  zero-padded  version  of  the 
spectrum  {Tm}^=1  of  the  frame  operator  FF*.  In  [1],  Antezana  el  al.  make  the  connection 
between  the  Schur-Horn  Theorem  and  majorization  explicit,  extending  the  Schur-Horn 
Theorem  to  give  conditions  on  the  existence  of  frames  with  prescribed  norms  and  frame 
operator  FF* .  This  connection  is  also  made  clear  in  [45].  Majorization  also  plays  a  role  in 
the  frame  completion  problem  and  is  used  to  determine  which  frames  can  be  built  from  a 
preexisting  set  of  vectors. 

Also  of  interest  is  the  problem  of  explicitly  constructing  frames  which  satisfy  the 
Schur-Horn  Theorem.  Algorithms  relying  on  Givens  rotations  [14,  15,  18,  19]  have  been 
used  to  produce  self-adjoint  matrices  with  a  given  majorized  diagonal.  Dhillon  el  al .,  in 
particular,  improve  upon  the  Chan-Li  [15]  and  Bendel-Mickey  [4]  algorithms  and  present 
more  generalized  versions  of  these  algorithms  which  have  the  ability  of  constructing  a 
much  larger  subset  of  the  corresponding  manifold  of  desired  matrices.  Chu  [16]  also 
considered  constructing  such  matrices  via  a  lift-and-projection  method  and  a  projected 
gradient  method.  Each  of  these  methods  provides  a  more  constructive  proof  of  Horn’s 
original  result  [32].  In  the  remainder  of  this  section,  we  discuss  interlacing  and  its  relevance 
to  majorization. 
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Interlacing  inequalities  arise  when  considering  the  relationship  among  eigenvalues 
of  principle  submatrices  [34,  49]  and  have  also  been  applied  to  matrices  associated  with 
graphs  [28].  Interlacing  is  also  useful  in  that  it  provides  an  alternative  way  of  viewing 
majorization  [33].  Here  we  consider  the  inverse  eigenvalue  problem  which  involves 
reconstructing  a  matrix  from  prescribed  spectral  data.  Inverse  eigenvalue  problems  not 
only  pose  spectral  constraints,  but  also  structural  constraints  upon  a  solution  matrix.  A 
survey  of  how  interlacing  applies  to  these  types  of  problems  can  be  found  in  [7],  and  more 
recently  in  [17]. 

For  our  research,  interlacing  comes  into  play  by  considering  partial  sums  of  the  frame 
operator  (3.4).  First  let  {A„;m)^=l  and  {An+Um}^=1  denote  the  spectrum  of  FnF*  and  Fn+lF*+v 
respectively.  Here,  Fn  denotes  the  Mxn  submatrix  of  the  MxN  matrix  F  obtained  by  taking 
its  first  n  columns.  By  applying  a  classical  result  in  [33]  to  Fn+]  F*+]  =  FnF*  +  fn+lf*+v 
where  fn+lf*+l  is  a  rank-one  positive  operator,  it  can  be  shown  that  {An+ interlaces 
with  {/l„;m}^=| .  Another  result  of  Horn  and  Johnson,  which  will  be  revisited  in  Chapters  3 
and  4,  states  that  given  sequences  { A„ }^= ,  and  ,  such  that  [d„}^=1  >  !/./„}')  ,,  there 
exists  a  sequence  {Xn}^=l  such  that  {d«}^=1  interlaces  with  {An}^  and  {d,,}^1  majorizes 

{ P3]. 
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III.  Constructing  frames  of  a  given  spectrum  and  set  of  lengths 


In  this  chapter,  we  provide  an  algorithm  for  how  to  explicitly  construct  every  frame 
whose  frame  operator  has  a  given  arbitrary  spectrum  and  whose  vectors  are  of  given 
arbitrary  lengths.  The  major  results  are  Theorem  2,  which  gives  a  two-step  algorithm  for 
constructing  all  such  frames,  and  Theorem  7,  which  makes  Step  B  of  Theorem  2  much 
more  explicit  and  implementable. 

In  order  to  construct  all  such  frames,  we  make  use  of  the  existing  theory  of 
majorization  and  the  Schur-Horn  Theorem.  Namely,  given  two  nonnegative  nonincreasing 
sequences  {An}^=  1  and  {p„}^=l,  recall  that  {A„}%=1  majorizes  {//„}",,  denoted  {An}^=l  > 

,  if 

n  n 

n'>Yj^n'  Vft  =  1,  .  .  .  ,iV  -  1,  (3.1) 

n'—\  n'= 1 

N  N 

X  An'  =  (3-2) 

n'= 1  n'= 1 

One  of  Schur’s  results  stated  that  the  spectrum  of  a  self-adjoint  positive  semidefinite  matrix 
necessarily  majorizes  its  diagonal  entries  [42].  Some  time  later,  Horn  showed  that  if 
{/t„}^=|  >  {pn}^=l,  then  there  exists  a  self-adjoint  matrix  that  has  {d„}^=1  as  its  spectrum  and 
{yu„}^=1  as  its  diagonal  [32].  Combining  these  two  results  gives  the  Schur-Horn  Theorem: 

Schur-Horn  Theorem.  There  exists  a  positive  semidefinite  self-adjoint  matrix  with 
spectrum  { An  ,  and  diagonal  entries  {pn}^={  if  and  only  if{An}^=l  >  {p„}^=]. 

By  applying  this  theorem  to  the  Gram  matrix  F*F,  whose  diagonal  entries  are  given 
by  {||/„||2}"  j  and  whose  spectrum  is  [T„}^=1,  we  are  able  to  determine  when  a  frame  with 
prescribed  spectrum  and  lengths  exists.  Here  it  is  important  to  note  that  the  spectrum  of  the 
Gram  matrix  is  a  zero-padded  version  of  the  spectrum  {Am\M=]  of  the  frame  operator  FF* . 
The  only  difference  between  the  eigenvalues  of  the  Gram  matrix  and  the  frame  operator  is 
zero  eigenvalues;  that  is,  A„  =  0  for  M  <  n  <  N. 
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For  the  current  research,  we  consider  an  alternative  approach  to  majorization  which 
involves  the  repeated  application  of  eigenvalue  interlacing  [33].  Recall  that  we  say  a 
nonnegative  nonincreasing  sequence  {y,„)^=|  interlaces  on  another  such  sequence  {/3„,}^=l , 
denoted  {/3m("=l  E  {y,„};Jf=l ,  provided 

Pm  <  jm  ^  Pm-  1  ^  yw-i  <  ■  ■  ■  <  Pi  <  yi  <  P\  <  yi  •  (3.3) 

Interlacing  arises  in  frame  theory  in  the  following  context:  given  any  sequence  of  vectors 
F  =  {/„}^=1  in  WM,  then  for  every  n  =  1, . . . ,  N,  we  consider  the  partial  sequence  of  vectors 
Fn  :=  {/„,}"  =1.  Note  that  FN  =  F  and  the  frame  operator  of  Fn  is 

n 

=  (3.4) 

n'= 1 

Let  {/l„;m}“=1  denote  the  spectrum  of  (3.4).  For  any  n  =  l, ...  ,N  -  l,  (3.4)  gives  that 
F)+1F*+1  =  FnF*  +  fn+lf*+l.  By  a  result  in  [33]  involving  the  addition  of  rank-one  positive 
operators  we  know  that  {An;m}^=1  E  {A„+1;m}m=i-  That  is,  the  spectrum  of  the  (n  +  l)st  partial 
sum  interlaces  on  the  spectrum  of  the  nth  partial  sum.  Moreover,  if  ||  /„||2  =  /./„  for  all 
n  =  1 ,...  ,N,  then  for  any  such  n, 

M  n  n 

Y  =  Tr (F„F*„)  =  Tr(F*F„)  =  Y  \\fA 1  =  Y^'  (3'5) 

m—  1  n'= 1  rc'=l 

Thus,  for  any  given  frame,  the  sequence  of  the  spectra  of  its  partial  frame  operators  satisfies 
a  set  of  interlacing  and  trace  conditions.  As  such,  in  order  to  construct  a  frame  F  with 
prescribed  spectrum  and  lengths,  we  give  a  name  to  sequences  of  interlacing  spectra  that 
satisfy  (3.5): 

Definition  1.  Given  nonnegative  nonincreasing  sequences  {Am}^=1  and  a  sequence 

of  outer  eigensteps  is  a  doubly-indexed  sequence  of  sequences  {{An;m}^=l)n=o  f°r  which: 

(i)  The  initial  sequence  is  trivial:  Ao;m  =  0,  Vm  =  1 , ...  ,M. 

(ii)  The  final  sequence  is  {A m}^=1:  AN:m  =  Am,  Vm  =  1, . . . ,  M. 
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(iii)  The  sequences  interlace:  U„_i;m)"=l  E  {An;m}%=1,  Vn= 

M  n 

(iv)  The  trace  condition  is  satisfied:  Fn',  Vn  =  l,...,N. 

m- 1  n'= 1 

We  refer  to  the  values  {{An-m\^=]  }^=0  as  outer  eigensteps  since  they  arise  from  sums 
of  outer  products  of  the  fjs  (frame  operators).  Every  sequence  of  vectors  whose  frame 
operator  has  the  spectrum  {Am}^=l  and  whose  vectors  have  squared  lengths  {pn}„=i  generates 
a  sequence  of  outer  eigensteps.  In  the  remainder  of  this  chapter,  we  show  that  the  converse 
is  also  true.  To  be  precise,  one  of  our  main  results,  Theorem  2,  provides  an  algorithm  for 
explicitly  constructing  every  possible  finite  frame  of  a  given  spectrum  and  set  of  lengths, 
in  terms  of  such  eigensteps. 


3.1  The  necessity  and  sufficiency  of  eigensteps 

In  this  section,  our  goal  is  to  prove  the  following  theorem: 

Theorem  2.  For  any  nonnegative  nonincreasing  sequences  {Am}^=1  and  {p„}„=l,  every 
sequence  of  vectors  F  =  {/„}^=1  in  whose  frame  operator  FF*  has  spectrum  {Am}^=1 
and  which  satisfies  \\fn\\2  =  Pnfor  all  n  can  be  constructed  by  the  following  process: 

A.  Pick  outer  eigensteps  {{/l„;m[^=I  }^=()  as  in  Definition  1. 


B.  For  each  n  =  1 , ,N,  consider  the  polynomial: 

M 

Pn(x)'-=\\(x- An-m).  (3.6) 

m- 1 


Take  any  f\  6  such  that  ||/i||2  =  p\.  For  each  n  =  1, . . . , N  -  1,  choose  any  fn+ 1 
such  that 


II  PnJr, 


n+ 1 1 


,.  .  n Pn+\(X ) 

-hm(x-d) — — 

Pn(x) 


(3.7) 


for  all  A  e  {An;m}^=l,  where  Pn;A  denotes  the  orthogonal  projection  operator  onto  the 
eigenspace  N(TI  -  FnF*)  of  the  frame  operator  Fn  F*  of  Fn  =  {fnfnn, =1.  The  limit 


in  (3.7)  exists  and  is  nonpositive. 
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Conversely,  any  F  constructed  by  this  process  has  {Am}^=l  as  the  spectrum  of  FF*  and 
||/„||2  =  Pnfor  all  n.  Moreover,  for  any  F  constructed  in  this  manner,  the  spectrum  of  FnF* 
is  U«;mC=,  for  all  n  =  1, . . . ,  N. 

Theorem  2  is  not  an  easily-implementable  algorithm  nor  does  it  address  the  existence 
of  such  an  F  for  a  given  {Am}^=l  and  {pn}^=v  Later  in  this  chapter,  we  will  address  Step  B 
and  how  to  compute  orthonormal  eigenbases  for  each  Fn.  In  the  following  chapter,  we 
discuss  Step  A  which  requires  one  choose  a  valid  sequence  of  eigensteps.  In  order  to  prove 
Theorem  2,  we  first  prove  the  following  lemma: 

Lemma  3.  Let  {Am}^=l  and  {pn}^=l  be  nonnegative  and  nonincreasing,  and  let  {{d„;m}^=1}^=0 
be  any  corresponding  sequence  of  outer  eigensteps  as  in  Definition  1.  If  a  sequence  of 
vectors  F  =  {/„}^=1  has  the  property  that  the  spectrum  of  the  frame  operator  FnF*  of 
Fn  =  {fn'Yf=i  is  for  all  n  =  1  then  the  spectrum  of  FF*  is  {Am}^=l  and 

WfnW2  =  Pnfor  all  n  =  1 

Proof  Definition  l.(ii)  immediately  gives  that  the  spectrum  of  FF*  =  FNF*N  is  indeed 
! A„i}^=  |  =  {/l/v;m(^=l ,  as  claimed.  Moreover,  for  any  n  =  1, . . . , N,  Definition  l.(iv)  gives 

n  M  n 

Y  Ill'll2  =  Tr(W  =  Tr (FnF*n)  =  Y^m  =  Y^'-  W 

n'= 1  m- 1  n'= 1 

Letting  n  =  1  in  (3.8)  gives  ||/i||2  =  p\,  while  for  n  =  2, . . . , N,  considering  (3.8)  at  both  n 
and  77-1  gives 

n  n- 1  n  n— 1 

WfnW2  =  Y  Hrf  -  Y  =  TjV"'  -  Y^’  =  u 

n'= 1  n'= 1  n'= 1  n'= 1 

The  next  result  gives  conditions  that  a  vector  must  satisfy  in  order  for  it  to  perturb  the 
spectrum  of  a  given  frame  operator  in  a  desired  way,  and  was  inspired  by  the  proof  of  the 
Matrix  Determinant  Lemma  and  its  application  in  [3]. 

Theorem  4.  Let  Fn  =  \f,')nn,=\  an  arbitrary  sequence  of  vectors  in  and  let  {An;m}^=1 
denote  the  eigenvalues  of  the  corresponding  frame  operator  FnF*n.  For  any  choice  of  fn+\ 
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in  M.m,  let  F,I+1  =  {//I'/Jp  Then  for  any  A  e  {A  n;m}^=l,  the  norm  of  the  projection  of  fn+i 
onto  the  eigenspace  N(/1I  -  FnF*n)  is  given  by 

l|FM/„+il|2  =  -lim(x-T)^^, 

pfx) 

where  pn(x)  and  pn+\(x)  denote  the  characteristic  polynomials  of  Fn F*  and  Fn+lF*+v 
respectively. 

Proof.  For  the  sake  of  notational  simplicity,  we  let  F  :=  Fn,  f  :=  fn+\  and  so  Fn+]  F*+|  = 
FF*  +  //*.  Suppose  *  is  not  an  eigenvalue  of  Fn+]F*+].  Then: 

Pn+i(x)  =  det(jcl  -  FF *  -  ././*) 

=  det(*I  -  FF*)det(I  -  (jcI  -  FF*)~X ff*) 

=  pn(x) det(I  -  (x\  -  FF*)-1//*).  (3.9) 

We  can  simplify  the  determinant  of  I  -  (jcI  -  FF*)-1//*  by  multiplying  by  certain  matrices 
with  unit  determinant: 


det(I-(xl-FF*)-1//*) 


=  det 


=  det 


=  det 


VL 


I  -  (x\  -  FF*)-1//* 

0 


I  0 

r  i_ 

I  0  I 

r  ij[-r 

I  -(xl  -  FF*)-1/ 

0  1  -  f  (xl  -  FF*)-1/ 

1  -  f*{x\  -  FF*)-1/. 


-(xl  -  FF*)-1/ 
1 


I  0 
-/*  1 


v 


-(xl  -FF*)-1/ 
1 


jy 


v 


(3.10) 


M 


We  now  use  (3.9)  and  (3.10)  with  the  spectral  decomposition  FF*  =  ^  A„;„ 


m=  1 


Pn+ l(x)  =  Pn(x)(  1  -  f*(x I  -  FF*)  lf )  =  pn(x)[  1  -  V  L 

A—i  r  — 


I  (f,Um)Y 


m=  1 


A'  -  /L 


(3.11) 
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Rearranging  (3.11)  and  grouping  the  eigenvalues  A  =  {An;m}^=1  according  to  multiplicity 
then  gives 


Pn+ l(x)  _ 
Pn(x) 

As  such,  for  any  A  e  A, 


M 


m- 1 


X  ^ n:.m 


‘-E 


/I'sA 


ll^;,r/H2 

x  -  A' 


Vjc  i  A. 


lim(.r  -  /t)^"+1^  =  lim(.r  -  /l)(l  -  'V 

x-**  p„(x)  ^ 


/I'eA 


Il/Wll2 

jc  -  /!' 


=  lim 

*— >/l 


(X  -A)-  \\Pn-J\\2  -  Yj  WPnwf\f 


A'?  A 


x  -  A 
x  -  A' 


=  -|l^;4/lP 


yielding  our  claim. 


□ 


While  the  following  two  lemmas  depend  only  on  basic  algebra  and  calculus,  their 
results  will  be  used  in  the  proof  of  Theorem  2. 

Lemma  5.  If  {J3m}%=1  and  {ym)^=l  are  real  and  nonincreasing,  then  {J3m}%=l  E  |ym};|f=l  if 
and  only  if 

lim  (x  -  <  0  Vm  =  1, . . . ,  M, 

x-^Pm  p(x) 

M  M 

where  p(x)  =  r>  -  f5m)  and  q(x)  =  IT1  Tm)* 

m=l  m=l 

Proof  (=>)  Let  {ym}^f=1  interlace  on  {J3m}^=],  and  let  T  =  /?,„  for  some  m  =  1, . . . ,  M.  Letting 
Lp  denote  the  multiplicity  of  A  as  a  root  of  pix),  the  fact  that  {[3m)™=]  E  {ym}^f=1  implies  that 
the  multiplicity  Lq  of  A  as  a  root  of  q(x)  is  at  least  Lp  —  1 .  Moreover,  if  Lq  >  Lp  —  1  then 
our  claim  holds  at  A  since 


lim(jc  -  d)-^— -  =  0  <  0. 

x->a  p(x) 

Meanwhile,  if  Lq  =  Lp  -  1 ,  then  choosing  mp  =  min {m  :  f3m  =  d}  gives 


Pm 

>  A, 

1  < 

m 

< 

mp  -  1, 

Pm 

=  A, 

VI 

a. 

s 

m 

< 

nip  +  Lp  1, 

Pm 

<A, 

TYlp  +  Lp  ^ 

m 

< 

M. 
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(3.12) 


We  now  determine  a  similar  set  of  relations  between  A  and  all  choices  of  ym .  For  1  <  m  < 
mp  -  1,  interlacing  and  (3.12)  imply  ym  >  (3m  >  A.  If  instead  mp  +  1  <  m  <  mp  +  Lp  -  1,  then 
interlacing  and  (3.12)  imply  A  =  fi„,  <  ym  <  J3m-i  =  A  and  so  ym  =  A.  Another  possibility  is 
to  have  mp  +  Lp  +  1  <  m  <  M,  in  which  case  interlacing  and  (3.12)  imply  ym  <  (3m_\  <  A. 
Taken  together,  we  have 


y m 

>4, 

1 

< 

m 

< 

nip 

ym 

=  4, 

mp  +  1 

< 

m 

< 

mp 

ym 

<4, 

nip  +  Lp  +  1 

< 

m 

< 

M. 

(3.13) 


Note  that  (3.13)  is  unlike  (3.12)  in  that  in  (3.13),  the  relationship  between  ym  and  A  is  still 
undecided  for  m  -  mp  and  m  =  mp  +  Lp.  Indeed,  in  general  we  only  know  ymp  >  fim  =  A, 
and  so  either  y„lp  =  A  or  ynip  >  A.  Similarly,  we  only  know  ym„+Lp  <  Pmp+Lp- i  =  T,  so  either 
7hip+l(,  =  A  or  y,„p+Lp  <  A.  Of  these  four  possibilities,  three  lead  to  either  having  Lq  =  Lp  +  1 
or  Lq  =  Lp,  only  the  case  where  yIHp  >  A  and  ymp+Lp  <  T  leads  to  our  current  assumption 
that  Lq  =  Lp  —  1.  As  such,  under  this  assumption  (3.13)  becomes 


(3.14) 


ym 

>4, 

1  < 

m 

< 

nip 

ym 

=  4, 

nip  +  1  < 

m 

< 

nip 

ym 

<4, 

mp  +  Lp  < 

m 

< 

M. 

We  now  prove  our  claim  using  (3.12)  and  (3.14): 

mp 

(x  -  A)Lp  P](.v  -  ym)  ]~~[  (x  -  ym) 


M 


lim(x  -  A)^-j—-  =  lim 

x->A  p(x)  x->A 


m=  1 


M 


(x  -  A)Lp  ]~ [  (x  - 1 3m )  Y\  (X  -  P>n) 

m=  1  m=mp+Lp 

mp  M 

I (4  -  7m)  f”[  (4  -  ym) 


m- 1 


M 


n  (4-js«)  n (/{-^ 


m—  1 


<  o. 


21 


(<=)  We  prove  by  induction  on  M.  For  M  -  1,  we  have  p(x)  =  x  -  (3[  and  q(x)  -  x  -  y i, 
and  so  if 


0  >  lim (x  ~  =  lim(oc-jSi)^ — ^  =  lim(jc-yi)  =  /?i  -yi, 

x— P\X)  x^Pi  \X — Pi)  x~‘ 

then  /?)  <  yi,  and  so  {y  1}  interlaces  on  {/?i },  as  claimed.  Now  assume  this  direction  of  the 
proof  holds  for  M'  =  1, . . . ,  M  -  1,  and  let  {/3m}^=l  and  {ym}ff=1  be  real,  nonincreasing  and 
have  the  property  that 


lim  (x  —  Pm)~~7~~  <  0  Vm  =  1, . . . ,  M,  (3.15) 

X~>Pm  p{X) 

where  p{x)  and  q(x)  are  defined  as  in  the  statement  of  the  result.  We  will  show  that  {ym}“=] 
interlaces  on  {/3m)"=|. 

To  do  this,  we  consider  two  cases.  The  first  case  is  when  {/?,„}„=!  and  {ym(^=l  have 
no  common  members,  that  is,  (3m  ±  ym>  for  all  in,  in'  =  1 , . . . ,  M.  In  this  case,  note 
that  if  J3m  -  fim>  for  some  m  ±  in'  then  the  corresponding  limit  in  (3.15)  would  diverge, 
contradicting  our  implicit  assumption  that  these  limits  exist  and  are  nonpositive.  As  such, 
in  this  case  the  values  of  {/Sm}^=l  are  necessarily  distinct,  at  which  point  (3.15)  for  a  given 
m  becomes: 


0> 


M 

lim  (x -/?,„)  fl 

m—  1 


(•^  Pm' ) 


M 

|  |  (Pm  Tm') 

m'= 1 
M 


j- [  (J3,n  ~P,n’) 

m'= 1 
m'tm 


M 

|  |  (Pm  Tm') 

m'= 1 

m-1  M 

H  _  Pm’ )  J- [  iPm  -  Pm’ ) 

m'=l  m'=m+l 


(3.16) 


Moreover,  since  /?,„  ±  ym,  for  all  in,  in',  then  the  limit  in  (3.16)  is  nonzero.  As  the  sign  of 


the  denominator  on  the  right-hand  side  of  (3.16)  is  (-l)m  1 ,  the  sign  of  the  corresponding 
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numerator  rs 


M 

(-1)"1  =  sgn^  (fim  -  7m')j  =  sgn(q(fim))  Vm  =  1 

m'= 1 

Thus,  for  any  m  =  2 q(x)  changes  sign  over  i],  implying  by  the 

Intermediate  Value  Theorem  that  at  least  one  of  the  roots  {ym}“=l  of  ry(.v)  lies  in  (fin„fim_x). 
Moreover,  since  q(x )  is  monic,  we  have  lim*-^  q(x)  =  oo;  coupled  with  the  fact  that 
q(j3 1)  <  0,  this  implies  that  at  least  one  root  of  q(x)  lies  in  (fi i,oo).  Thus,  each  of  the 
M  disjoint  subintervals  of  (fix,  oo)  u  [u^=9(/?„„  fi,n-i)\  contains  at  least  one  of  the  M  roots  of 
q{x).  This  is  only  possible  if  each  of  these  subintervals  contains  exactly  one  of  these  roots. 
Moreover,  since  {ym}^=1  is  nonincreasing,  this  implies  fix  <  y\  and  fim  <  ym  <  fi,„_x  for  all 
m  =  2  meaning  that  {ym}^f=1  indeed  interlaces  on  {fi,n\^=] . 

We  are  thus  left  to  consider  the  remaining  case  where  {fim}^=1  and  {y,„}^=|  share  at  least 
one  common  member.  Fix  A  such  that  fim  =  A  =  yIH>  for  at  least  one  pair  m,  in'  =  1 , ,M. 
Let  nip  =  min {m  :  fim  =  2}  and  mq  =  min{»r  :  y,„  =  d}.  Let  Fix)  and  Q(x)  be  (M  -  1)- 
degree  polynomials  such  that  p(x)  =  (x  -  A)P(x)  and  q(x)  -  (x  -  A)Q(x).  Here,  our 
assumption  (3.15)  implies 


0  >  lim  {x  -fim) 

X 


q(x) 

p{x) 


lim  ( x-fim ) 

X 


(x-A)Q(x) 
(x  -  A)P(x) 


lim  ( x-fi,„ ) 

X 


Q(x) 

P(x) 


(3.17) 


for  all  m  =  1  Since  P(x)  and  Q(x)  satisfy  (3.17)  and  have  degree  M  —  1,  our 

inductive  hypothesis  gives  that  the  roots  {ym}m±mq of  Q(x)  interlace  on  the  roots  {fim}m±mp  of 
P(x). 

We  claim  that  mq  is  necessarily  either  mp  or  mp  +  1,  that  is,  mp  <  mq  <  mp  +  1.  We 
first  show  that  mp  <  mq,  a  fact  which  trivially  holds  for  mp  =  1.  For  mp  >  1,  the  fact  that 
{fim)mtm„  E  {ym\m+mq  implies  that  the  value  of  the  (mp  -  l)th  member  of  {ym}m±mq  is  at  least 
that  of  the  (mp  -  l)th  member  of  {fim}m±mp.  That  is,  the  (mp  -  l)th  member  of  {ym}m±mq  is  at 
least  fimp-\  >  A,  meaning  mp  -  1  <  mq  -  1  and  so  mp  <  mq,  as  claimed.  We  similarly  prove 
that  mq  <  mp  +  1,  a  fact  which  trivially  holds  for  mp  =  M.  For  mp  <  M,  interlacing  implies 
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that  the  mp th  member  of  {pm\m±m  is  at  least  the  (mp  +  l)th  member  of  {y„ t}m±mq-  That  is, 
the  {mp  +  l)th  member  of  \ym\m±mq  is  at  most  pmp+\  <  A  and  so  mp  +  1  >  mq,  as  claimed. 
Now,  in  the  case  that  mq  =  mp,  the  fact  that  {J3m}m*mp  E  {ym\m±mq  implies  that 

Pm  <  7m  ^  ^  Pm„+\  ^  7mp+l  ^  Pmp- 1  ^  7mp- 1  <  '  '  '  <  Pi  <  7l-  (3.18) 

Since  in  this  case  y,„p+ 1  =  ym?+i  <  d  <  /?mp_i,  the  terms  =  d  and  y,„p  =  y,„9  =  d  can  be 
inserted  into  (3.18): 

Pm  <  7m  <  •  •  •  <  Anp+1  ^  7mp+l  <  d  <  d  <  Pmp-\  <  7mp-l  <  •  •  •  <  /?1  <  7l, 

and  so  {P,n}^=l  E  {ym}m=i-  In  the  remaining  case  where  mq  =  +  1,  having  E 

{7m}m^m?  means  that 

Pm  —  7m  —  '  '  '  —  Pmp+ 2  —  y ntp+2  —  Pmp+ 1  —  7mp  —  Pmp- 1  —  7mp- 1  —  '  '  '  —  I  —  7l  •  (3.19) 

Since  in  this  case  y3mp+,  <  d  <  ym?_i  =  ymp,  the  terms  ymp+i  =  ym<?  =  d  and  pmp  =  d  can  be 

inserted  into  (3.19): 


Pm  —  yM  —  "  "  "  —  7m„+ 2  —  Pm„+ 1  —  d  <  d  <  7m.,  —  Pin,,- 1  —  '  '  '  —  /h  —  7l 


and  so  {/3m}“=1  E  {y„,}“=1  in  this  case  as  well. 


Lemma  6.  Jf{pm\m=\’  {7mlm=i’  an^  rea^ an^  nonincreasing  and 


□ 


lim  (x  -  /3m)  — —  =  lim  (x  -  /?,„)  — - 

x^>/3m  p(X)  x  >/3m  p(x) 


dm  =  1  ,...,M, 


M 


M 


M 


where  p{x)  =  J  J (x  -  pm),  q(x)  =  J  J(x  -  ym)  and  r(x)  =  [(x  -  8m),  then  q(x)  =  r(x). 

m= 1  m=l  m=l 

Proof.  Fix  any  m  =  l, ...  ,M,  and  let  L  be  the  multiplicity  of  pm  as  a  root  of  p(x).  Since 

(3.20) 


r  /  a  x^W  r  ,  0  /W 

lim  (x  -  pm)  ——  =  lim  (x  -  pm)  ——  , 

*->An  p(x)  x  p(x) 


where  each  of  these  two  limits  is  assumed  to  exist,  then  the  multiplicities  of  pm  as  a  roots 
of  q(x)  and  r(x)  are  both  at  least  L  -  1.  As  such,  evaluating  /th  derivatives  at  pm  gives 


24 


q(l>(j3m)  =  0  =  ril)(J3m )  for  all  /  =  0, . . .  ,L  -  2.  Meanwhile,  for  l  =  L  -  1,  l’Hopital’s  Rule 


gives 


lim  (x-j3m) 

X  >f3m 


q(x) 

p(x) 


q(x )  (x-/3w)l 

lim  - — - 

(x  -  Pm)L  p(x) 

q{L~Y)(J3m)  L\ 

(L-l)!  pV> (J3m) 

Lg(L-Y)ifim) 

P(L)(fim) 


(3.21) 


Deriving  a  similar  expression  for  r(x)  and  substituting  both  it  and  (3.21)  into  (3.20)  yields 
q(L~l>(fim)  =  r(L~l>(fim).  As  such,  q(l)(J3m)  =  ra>(J3m)  for  all  /  =  0, . . . ,  L  -  1.  As  this  argument 
holds  at  every  distinct  we  see  that  q(x)  -  r(x)  has  M  roots,  counting  multiplicity.  But 
since  q(x)  and  r(x)  are  both  monic,  q(x)-r(x)  has  degree  at  most  M- 1  and  so  q(x)-r(x)  =  0, 
as  claimed.  □ 


With  Theorem  4  and  Lemmas  3,  5  and  6  in  hand,  we  are  ready  to  prove  the  main  result 
of  this  section. 

Proof  of  Theorem!.  (=>)  Let  {/t,„}^=1  and  {pn}„=i  be  arbitrary  nonnegative  nonincreasing 
sequences,  and  let  F  =  {f,}^=i  be  any  sequence  of  vectors  such  that  the  spectrum  of  FF* 
is  {fn}w=l  and  \\f,\\ 2  =  pn  for  all  n  =  1  We  claim  that  this  particular  F  can  be 

constructed  by  following  Steps  A  and  B. 

In  particular,  consider  the  sequence  of  sequences  {{/l„;m}^=|  \Nn=()  defined  by  letting 
{An\m\m=i  be  the  spectrum  of  the  frame  operator  FnF*  of  the  sequence  Fn  =  =1  for 

all  77  =  1  and  letting  A  0;m  =  0  for  all  m.  We  claim  that  \\Amm\^=]  }^0  satisfies 

Definition  1  and  therefore  is  a  valid  sequence  of  outer  eigensteps.  Note  conditions  (i) 
and  (ii)  of  Definition  1  are  immediately  satisfied.  To  see  that  {{An-m}^=l}^=0  satisfies  (iii), 
consider  the  polynomials  p„(x)  defined  by  (3.6)  for  all  n  -  1, . . . ,  N.  In  the  special  case 
where  n  =  1,  the  desired  property  (iii)  that  {0}'^=|  E  {d,;m}^=|  follows  from  the  fact  that  the 
spectrum  {Ai-m}ffi=l  of  the  scaled  rank-one  projection  F]  F*  =  f  f  *  is  the  value  ||/i|]2  =  p\ 
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along  with  M  —  1  repetitions  of  0,  the  eigenspaces  being  the  span  of  f\  and  its  orthogonal 
complement,  respectively.  Meanwhile  if  n  =  2  Theorem  4  gives  that 


Pn(x)  _ 

lim  (.V  I  ;m )  .  .  \\P 

X  ’.(ji  ■  ]  'jn  |  {  .V ) 


implying  by  Lemma  5  that  {T„_  i;m}„=1  E  {T„. 


lM 

■mhn=\ 


fn\ l2<0  Vm  =  1, . . . ,  M, 

as  claimed.  Finally,  (iv)  holds  since  for 


any  n  =  l,...,iV  we  have 

M  n  n 

Y  =  Tr (FnK)  =  Tr (F*nFn)  =  Y  WfA\2  =  Y^' 

m=  1  n'- 1  n'  - 1 

Having  shown  that  these  particular  values  of  {{T„;m}^=1}^=0  can  indeed  be  chosen  in 
Step  A,  we  next  show  that  our  particular  F  can  be  constructed  according  to  Step  B.  As  the 
method  of  Step  B  is  iterative,  we  use  induction  to  prove  that  it  can  yield  F.  Indeed,  the  only 
restriction  that  Step  B  places  on  fx  is  that  ||/i  ||2  =  /ii ,  something  our  particular  f\  satisfies  by 
assumption.  Now  assume  that  for  any  n  =  1, . . . ,  N  -  1  we  have  already  correctly  produced 
{/«'}"'= i  by  following  the  method  of  Step  B;  we  show  that  we  can  produce  the  correct  fn+l 
by  continuing  to  follow  Step  B.  To  be  clear,  each  iteration  of  Step  B  does  not  produce  a 
unique  vector,  but  rather  presents  a  family  of  /„+ i’s  to  choose  from,  and  we  show  that  our 
particular  choice  of  fn+\  lies  in  this  family.  Specifically,  our  choice  of  /„+,  must  satisfy  (3.7) 
for  any  choice  of  A  6  {An;m}^=l ;  the  fact  that  it  indeed  does  so  follows  immediately  from 
Theorem  4.  To  summarize,  we  have  shown  that  by  making  appropriate  choices,  we  can 
indeed  produce  our  particular  F  by  following  Steps  A  and  B,  concluding  this  direction  of 
the  proof. 

(<=)  Now  assume  that  a  sequence  of  vectors  F  =  {/„)^=|  has  been  produced  according 
to  Steps  A  and  B.  To  be  precise,  letting  {{Tn;m}^=1}^=0  be  the  sequence  of  outer  eigensteps 
chosen  in  Step  A,  we  claim  that  any  F  =  {fn}^=l  constructed  according  to  Step  B  has  the 
property  that  the  spectrum  of  the  frame  operator  FnF*  of  F„  =  {/„>}", =1  is  {An;m}^=1  for  all 
n  =  1, . . . ,  N.  Note  that  by  Lemma  3,  proving  this  claim  will  yield  our  stated  result  that  the 
spectrum  of  FF*  is  {dm(^=|  and  that  \\fn\\2  =  /y„  for  all  n  =  1, . . . ,  N.  As  the  method  of  Step  B 
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is  iterative,  we  prove  this  claim  by  induction.  Step  B  begins  by  taking  any  f\  such  that 
||/i||2  -  p  i.  As  noted  above  in  the  proof  of  the  other  direction,  the  spectrum  of  FXF\  =  fxf* 
is  the  value  p\  along  with  M  —  1  repetitions  of  0.  As  claimed,  these  values  match  those  of 


l  M 


to  see  this’  note  that  Definition  l.(i)  and  (iii)  give  {0}”=1  =  {T0;m},„=1  E  Ui;mtm=1 
and  so  Ai;m  =  0  for  all  m  =  2, . . . ,  M,  at  which  point  Definition  l.(iv)  implies  A^\  =  Hi- 
Now  assume  that  for  any  n  =  1, . . . ,  N  -  1,  the  Step  B  process  has  already  produced 
Fn  =  =1  such  that  the  spectrum  of  FnF*  is  {A„;m}^=1.  We  show  that  by  following 

Step  B,  we  produce  an  fi+i  such  that  Fn+ 1  =  };'+J,  has  the  property  that  {An+ is 

the  spectrum  of  Fn+lF*+v  To  do  this,  consider  the  polynomials  pn(x)  and  pn+\(x)  defined 
by  (3.6)  and  pick  any  fn+\  that  satisfies  (3.7),  namely 


lim  (x  -  An;m)P"+ff  =  -\\Pn-An,J,+i\\2 

X  *^n;m 


Mm  =  1  ,...,M. 


(3.22) 


Letting  {An+ i;m}„=1  denote  the  spectrum  of  Fn+1F*+V  our  goal  is  to  show  that  {An+1;m,m=1  - 
{An+\,m}n=v  Equivalently,  our  goal  is  to  show  that  pn+\(x)  =  pn+\{x)  where  pn+i(x )  is  the 
polynomial 

M 

Pn+ i(a)  |  j(-^  /hi+l;m)> 


m—  1 


Since  pn(x)  and  pn+i(x)  are  the  characteristic  polynomials  of  FnF*  and  Fn+lF*+l, 
respectively,  Theorem  4  gives: 


lim  (x-An.m)— ^  =  -\\Pir,An,mfn+i\\2 

x  I?  f  i  (  X) 


Mm  =  1  ,...,M. 


(3.23) 


Comparing  (3.22)  and  (3.23)  gives: 

r  /  i  \P»+ i(*)  r  ,  3  J«+iW 

lini  \X  A n\m )  ,  v  —  liiri  (x  An-m) 

X  n\m  PfjyX)  X  Pn\^) 


Mm  =  1 


implying  by  Lemma  6  that  pn+i(x)  =  pn+i(x),  as  desired. 


□ 


3.2  Constructing  frame  elements  from  eigensteps 

In  the  previous  section,  we  proved  that  Theorem  2  provides  a  two-step  process  for 
constructing  any  and  all  sequences  of  vectors  F  =  {/„}^=1  in  WM  whose  frame  operator 
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possesses  a  given  spectrum  {/lm}“=|  and  whose  vectors  have  given  lengths  {pn\Nn=\-  In  this 
section,  we  focus  on  improving  Step  B,  namely  how  we  can  explicitly  construct  any  and 
all  sequences  of  vectors  whose  partial-frame-operator  spectra  match  the  outer  eigensteps 
chosen  in  Step  A.  While  the  algorithm  below  appears  very  technical,  it  can  nonetheless  be 
performed  by  hand  or  numerically  using  programs  such  as  MATLAB.  MATLAB  code  for 
implementing  this  algorithm  has  been  included  in  the  appendix. 

Theorem  7.  For  any  nonnegative  nonincreasing  sequences  {Am}^=1  and  {pn)„=v  every 
sequence  of  vectors  F  -  {fn}^=l  in  whose  frame  operator  FF*  has  spectrum  {Am}^=1 
and  which  satisfies  \\f,\\2  =  Pnfor  all  n  can  be  constructed  by  the  following  algorithm: 

A.  Pick  outer  eigensteps  {{A„-m}^=]  }^=0  as  in  Definition  1. 

B.  Let  U i  be  any  unitary  matrix,  U i  =  {u\-m)^i=],  and  let  f\  =  \[p\U\  \.  For  each 

n  =  1; 

B.l  Let  V„  be  an  M  X  M  block-diagonal  unitary  matrix  whose  blocks  correspond  to 
the  distinct  values  of  {An-m\^i=]  with  the  size  of  each  block  being  the  multiplicity 
of  the  corresponding  eigenvalue. 

B.2  Identify  terms  which  are  common  to  both  {An;m}^=1  and  {/t„+i;m}£f=1.  Specifically: 

•  Let  I  n  c  {1 , ,M)  consist  of  those  indices  m  such  that  An.m  <  Anin>  for 
all  in'  <  m  and  such  that  the  multiplicity  of  An-m  as  a  value  in  {An;m>}^/=] 
exceeds  its  multiplicity  as  a  value  in  {An+ 

•  Let  S',,  c  {1, . . . ,  M }  consist  of  those  indices  m  such  that  An+\rn  <  An+l.m'for 
all  m'  <  m  and  such  that  the  multiplicity  of  An-m  as  a  value  in  {An+i;mf^,=1 
exceeds  its  multiplicity  as  a  value  in  {A„;mf^,=1. 

The  sets  In  and  ffn  have  equal  cardinality,  which  we  denote  Rn.  Next: 
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•  Let  7Tjn  be  the  unique  permutation  on  {1, ... ,  M\  that  is  increasing  on  both 
I n  and  I\ \  and  such  that  njn(m )  G  { 1, . . . ,  Rn}for  cdl  m  e  I n.  Let  II jn  be  the 
associated  permutation  matrix  II jn6m  = 

•  Let  7Tjn  be  the  unique  permutation  on  {1, ... ,  M }  that  is  increasing  on  both 
ffn  and  and  such  that  njn(m)  G  {1, . . .  ,R„}  for  cdl  m  G  ffn.  Let  II jn  be 
the  associated  permutation  matrix  II  jn6m  =  6njn(m). 


B.3  Let  vn,  wn  be  the  RnX  1  vectors  whose  entries  are 

j- J  ( dnjn  dn+im"') 


vn(ni(m))  = 


m"€jn 


j  j  (d„-m  'Lrjn"  ) 


m"€j„ 

m" 


dm  G  I„ 


J  J  ^ n;m .") 


m"sj„ 


|~ J  ( d>t+  I  \m!  dn+  |  ;/r;" ) 

m”ej„ 

L  m"  Cm' 


dm'  G  Jn. 


B.4  fn+i  =  unvn nj 

n 

zeros. 


,  where  the  M  X  1  vector 


is  vn  padded  with  M  -  Rn 


B.5  Un+1  =  UnVnli 


Wn  0 
0  I 


II jn  where  W„  is  the  Rn  xRn  matrix  whose  entries  are: 


Wn(nIn(m),nJn(m'))  = 


1 


4  n+ 1  :m'  'brj 


-v„(n i„(m))wn(nJn(m')). 


Conversely,  any  F  constructed  by  this  process  has  {Am}^=]  as  the  spectrum  of  FF*  and 
WfnW2  =  Pn  for  all  n. 

Moreover,  for  any  F  constructed  in  this  manner  and  any  n  =  l, ...  ,N,  the  spectrum  of  the 
frame  operator  FnF*  arising  from  the  partial  sequence  Fn  =  {./«'  }'',= ,  is  {An;m}^=1,  and  the 
columns  of  Un  form  a  corresponding  orthonormal  eigenbasis  for  FnF*. 
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Before  proving  Theorem  7,  we  give  an  example  of  its  implementation,  with  the  hope 
of  conveying  the  simplicity  of  the  underlying  idea,  and  better  explaining  the  heavy  notation 


used  in  the  statement  of  the  result. 

Example  8.  We  now  use  Theorem  7  to  construct  UNTFs  consisting  of  5  vectors  in  M3. 
Here,  A\  =  A2  -  d3  =  f  and  =  /u2  =  n2  =  yu4  =  /u5  =  1.  By  Step  A,  our  first  task  is  to  pick 
a  sequence  of  outer  eigensteps  consistent  with  Definition  1,  that  is,  pick  {A\;i,  Ai;2,  Ai^}, 
{/t2;i,  T2;2,  d2;3 1 ,  {^3;!,  Ai;2,  T3;3}  and  {d4;i,  T4;2,  d4;3}  that  satisfy  the  interlacing  conditions: 


{0,0,0}  E  {Tl;l,  /ll;2,  >Ti;3  } 

{dl;l,  di;2,  /li;3}  E  {A2:i,  A2-2,  A2-2} 

{A2-i,  A2-2,  A2:s}  E  {d3;i,  d3;2,  T3;3}  (3.24) 

R3;l>  ^3;2>  ^3;3)  E  {T4;i,  /t4;2,  T4;3} 

{T4;l,  T4;2,  d4'3 }  E  {f ,  |,  |}, 

as  well  as  the  trace  conditions: 


^l;l  +  M;: 

l  +  4l;3 

;  =  1, 

42;l  +  A  2;; 

l  +  ^2;3  -  2, 

(3.25) 

^3:1  +  ^3; 

2  +  ^3;: 

3=3, 

d4;i  +  44; 

2  +  ^4:3  =  4. 

Writing  these  desired  spectra  in  a  table: 

77  0  1  2  3  4  5 

d„;  3  0  ?  ?  ?  ?  f 

A,,-,  0  ?  ?  ?  ?  f 

4,0  9  ?  ?  ?  5 

/Ln;l  \j  ...  .  3 

the  trace  condition  (3.25)  means  that  the  sum  of  the  values  in  the  nth  column  is  £",=1  TV  = 
77,  while  the  interlacing  condition  (3.24)  means  that  any  value  An;m  is  at  least  the  neighbor 
to  the  upper  right  An+  i-m+i  and  no  more  than  its  neighbor  to  the  right  4„+i;m.  In  particular, 


30 


for  n  =  1,  we  necessarily  have  0  =  d0;2  <  A1;2  <  /to;i  =  0  and  0  =  d0; 3  <  d1;3  <  d0; 2  =  0 
implying  that  A 1;2  =  di;3  =  0.  Similarly,  for  n  =  4,  interlacing  requires  that  |  =  d5;2  < 
d4 ■  ]  <  d.5;i  =  |  and  |  =  d5;3  <  /l4;2  <  d5;2  =  f  implying  that  /l4;i  =  d4;2  =  §.  That  is,  we 
necessarily  have: 


n 

0 

1 

2 

3 

4 

5 

n;3 

0 

0 

? 

? 

? 

5 

3 

n;2 

0 

0 

? 

? 

5 

3 

5 

3 

^n;l 

0 

? 

? 

? 

5 

3 

5 

3 

Applying  this 

same  idea  again  for 

n  = 

2  and  n 

=  3  gives  0  =  d  1  ;3 

<  d2;3  <  d  1  "2  =  0  and 

|  =  d4;2  <  d3;1 

<  d4;l  - 

|,  and  so  we  also  necessarily  have  that  d2;3 

=  0,  and  d3;i  =  |: 

n 

0 

1 

2 

3  4 

5 

^n;  3 

0 

0 

0 

?  ? 

5 

3 

^ n;2 

0 

0 

? 

?  5 

3 

5 

3 

^n;  1 

0 

? 

? 

5  5 

3  3 

5 

3 

Moreover,  the  trace  condition  (3.25)  at  n  =  1  gives  1  =  dip  +  d1;2  +  d1;3  =  d1;1  +0  +  0  and 
so  A.  1 ;  1  =  1.  Similarly,  the  trace  condition  at  n  =  4  gives  4  =  d4;i  +  d4;2  +  d4;3  =  |  +  |  +  d4;3 
and  so  d4;3  = 


n  0  1  2  3  4  5 

d„;3  0  0  0  ?  f  f 

d„;2  0  0  ?  ?  f  f 

4;t  0  1  ?  |  |  f 

The  remaining  entries  are  not  fixed.  In  particular,  we  let  d3;3  be  some  variable  +  and  note 
that  by  the  trace  condition,  3  =  d3;i  +  d3;2  +  d3;3  =  x  +  d3;2  +  f  and  so  d3;2  =  |  -  x.  Similarly 
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letting  A2; 2  =  y  gives  A2:\  =2  -y: 


0 


'C;! 


0 

0 

0 


0 

0 

1 


0 

y 

2 -y 


x 

4 


(3.26) 


We  take  care  to  note  that  x  and  y  in  (3.26)  are  not  arbitrary,  but  instead  must  be  chosen  so 
that  the  interlacing  relations  (3.26)  are  satisfied.  In  particular,  we  have: 


{^3;0  ^3:2>  ^3;3 

l}  E  {^4;!,  A4-2,  ^4;3 } 

x<  |  <  \-x<  f. 

{■^2;1>  ^2;2;  d2;3 

.(  E  {A^i,  A^t2,  A^} 

0<x<y<  |-x<2-y<  |, 

(3.27) 

di;3 

i(  E  {/t2;l,T2;2,/l2;3} 

0<y<l<2-y. 

By  plotting  each  of  the  11  inequalities  of  (3.27)  as  a  half-plane  (Figure  3.1(a)),  we  obtain  a 
5-sided  convex  set  (Figure  3.1(b))  of  all  (x, y)  such  that  (3.26)  is  a  valid  sequence  of  outer 
eigensteps.  Specifically,  this  set  is  the  convex  hull  of  (0,  |),  (|,  0,  (|,  |),  (|,  1)  and  (0,  |). 
We  note  that  though  this  analysis  is  straightforward  in  this  case,  it  does  not  easily  generalize 
to  other  cases  in  which  M  and  N  are  large. 


To  complete  Step  A  of  Theorem  7,  we  pick  any  particular  (x,y)  from  the  set  depicted 
in  Figure  3.1(b).  For  example,  if  we  pick  (x,y)  =  (0,  |)  then  (3.26)  becomes: 


0 


1 


'k:3 

2 

Trl 


0 

0 

0 


0 

0 

1 


0 


0 


(3.28) 


We  now  perform  Step  B  of  Theorem  7  for  this  particular  choice  of  outer  eigensteps.  First, 
we  must  choose  a  unitary  matrix  U\.  Considering  the  equation  for  Un+]  along  with  the  fact 


32 


Figure  3.1:  Pairs  of  parameters  (jqy)  that  generate  a  valid  sequence  of  outer  eigensteps 
when  substituted  into  (3.26).  To  be  precise,  in  order  to  satisfy  the  interlacing  requirements 
of  Definition  1,  x  and  y  must  be  chosen  so  as  to  satisfy  the  11  pairwise  inequalities 
summarized  in  (3.27).  Each  of  these  inequalities  corresponds  to  a  half-plane  (a),  and  the 
set  of  (x,y)  that  satisfy  all  of  them  is  given  by  their  intersection  (b).  By  Theorem  7,  any 
corresponding  sequence  of  outer  eigensteps  (3.26)  generates  a  3  x  5  UNTF  and  conversely, 
every  3x5  UNTF  is  generated  in  this  way.  As  such,  x  and  y  may  be  viewed  as  the  two 
essential  parameters  in  the  set  of  all  such  frames. 


that  the  columns  of  Un  will  form  an  eigenbasis  for  F,  we  see  that  our  choice  for  U i  merely 
rotates  this  eigenbasis,  and  hence  the  entire  frame  F,  to  our  liking.  We  choose  U\  =  I  for 
the  sake  of  simplicity.  Thus, 


fi  =  y/lAUvA  = 


1 

0 

0 


We  now  iterate,  performing  Steps  B.l  through  B.5  for  n  =  1  to  find  f2  and  U2,  then 
performing  Steps  B.l  through  B.5  for  n  =  2  to  find  /3  and  U 3,  and  so  on.  Throughout 
this  process,  the  only  remaining  choices  to  be  made  appear  in  Step  B.l.  In  particular, 
for  n  -  1  Step  B.l  asks  us  to  pick  a  block-diagonal  unitary  matrix  V\  whose  blocks  are 
sized  according  to  the  multiplicities  of  the  eigenvalues  {Ti;i,  Ti;2,  Ti;3}  =  {1,0,0}.  That 
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is,  V\  consists  of  a  1  x  1  unitary  block — a  unimodular  scalar — and  a  2  x  2  unitary  block. 
There  are  an  infinite  number  of  such  Vi’s,  each  leading  to  a  distinct  frame.  For  the  sake  of 
simplicity,  we  choose  V\  =  I.  Having  completed  Step  B.l  for  n  =  1,  we  turn  to  Step  B.2, 
which  requires  us  to  consider  the  columns  of  (3.28)  that  correspond  to  n  =  1  and  n  =  2: 


1  2 


An-  3  0  0 


(3.29) 


d„;  2  0  3 

An;  1  1  f 

In  particular,  we  compute  a  set  of  indices  I\  c  {1,2,3}  that  contains  the  indices  m  of 
{/li;i ,  d i;2,  di;3 }  =  {1,0,0}  for  which  (i)  the  multiplicity  of  Ai:m  as  a  value  of  {1,0, 0}  exceeds 
its  multiplicity  as  a  value  of  {A2;i,  A2-2,  A2;3i  =  if.  |,0}  and  (ii)  m  corresponds  to  the  first 
occurrence  of  A\-m  as  a  value  of  {1,0,0};  by  these  criteria,  we  find  1 1  =  {1,2}.  Similarly 
m  e  J~\  if  and  only  if  m  indicates  the  first  occurrence  of  a  value  A2;m  whose  multiplicity 
as  a  value  of  {f,  |,0}  exceeds  its  multiplicity  as  a  value  of  {1,0,0},  and  so  37 1  =  {1,2}. 
Equivalently,  I \  and  37 1  can  be  obtained  by  canceling  common  terms  from  (3.29),  working 
top  to  bottom;  an  explicit  algorithm  for  doing  so  is  given  in  Table  3.2  near  the  end  of  this 
chapter. 

Continuing  with  Step  B.2  for  n  =  1,  we  now  find  the  unique  permutation  nIx  : 
{1,2,3}  — >  {1,2,3}  that  is  increasing  on  both  1 1  =  {1,2}  and  its  complement  I\  -  {3} 
and  takes  I \  to  the  first  R\  =  |i~i|  =  2  elements  of  {1,2,3}.  In  this  particular  instance, 
7Tj,  happens  to  be  the  identity  permutation,  and  so  n =  I.  Since  37i  =  {1,2}  =  I we 
similarly  have  that  njx  and  ITj,  are  the  identity  permutation  and  matrix,  respectively. 

For  the  remaining  steps,  it  is  useful  to  isolate  the  terms  in  (3.29)  that  correspond  to  I \ 
and  37b 


Pi  -  A 1  ;2  -  0,  y2  -  A2-2  - 

/7  =  A 1 ;  1  =  1,  j\  =  A2-i  =  |. 


(3.30) 


34 


In  particular,  in  Step  B.3,  we  find  the  /i|  x  1  =2x1  vector  vi  by  computing  quotients  of 
products  of  differences  of  the  values  in  (3.30): 

[vi(l)]2  = 

[vi(2)]2  = 


yielding  vj 


2 

3 

VI 

L  3  J 


(fi\  ~7i)(fii  ~  7i) 

(1  -  f)(l  -  5)  4 

(3.31) 

(Pi -Pi) 

(1-0)  9’ 

(pi  -  yi)(Pi  -  yi) 

(0-f)(0-i)_5 

(3.32) 

(Pi- Pi) 

_V5 

(0-1)  9’ 

we  compute  wi  = 

V6 

1 

LVeJ 

according  to  the  formulas: 

(n  -Pi)(y i  -  Pi) 

_  ( 

f-DCf-O)  5 

(3.33) 

(n  -  72) 

d-i) 

(72  -pi)(yi  -  Pi) 

_  ( 

5  -  ixj  -  °)  , 

(3.34) 

(72-71) 

1 

1 

CM 

[Wl(l)]2  = 

[wi(2)]2  = 

Next,  in  Step  B.4,  we  form  our  second  frame  element  f2  =  U\V\  fl] 


1  0  0  1  0  0  1  0  0 

fi  =010010010 
0  0  1  0  0  1  0  0  1 

As  justified  in  the  proof  of  Theorem  7,  the  resulting  partial  sequence  of  vectors 

1 


V5 

3 

0 


V5 

3 

0 


fl  fl 


0 


2 

3 

V5 


3 

0  0 


has  a  frame  operator  F2F*  whose  spectrum  is  {A2;i,  A2t2,  A.2,3}  =  {f,|,0}.  Moreover,  a 
corresponding  orthonormal  eigenbasis  for  F0F *  is  computed  in  Step  B.5;  here  the  first  step 
is  to  compute  the  x  Ri  =2x2  matrix  W\  by  computing  a  pointwise  product  of  a  certain 
2x2  matrix  with  the  outer  product  of  vi  with  wq : 
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Note  that  W i  is  a  real  orthogonal  matrix  whose  diagonal  and  subdiagonal  entries  are  strictly 
positive  and  whose  superdiagonal  entries  are  strictly  negative;  one  can  easily  verify  that 
every  Wn  has  this  form.  More  significantly,  the  proof  of  Theorem  7  guarantees  that  the 
columns  of 


u2  =  f/1v1nji 
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form  an  orthonormal  eigenbasis  of  F2 F^ .  This  completes  the  n  =  1  iteration  of  Step  B;  we 
now  repeat  this  process  for  n  =  2,3,4.  For  n  =  2,  in  Step  B.l  we  arbitrarily  pick  some 
3x3  diagonal  unitary  matrix  V2.  Note  that  if  we  wish  our  frame  to  be  real,  there  are  only 
23  =  8  such  choices  of  V2.  For  the  sake  of  simplicity,  we  choose  V?  =  I  in  this  example. 
Continuing,  Step  B.2  involves  canceling  the  common  terms  in 


to  find  I2  -  J2  -  {2},  and  so 


n  2  3 


3  0  0 


4 /j;2 


1 

3 


4 

3 


4/j;! 


5 

3 


5 

3 


n j2  —  fIto  - 


x 


o  1  o 
1  o  o 
o  o  1 


36 


In  Step  B.3,  we  find  that  v2  =  w2  =  1  .  Steps  B.4  and  B.5  then  give  that  F3  =  J\  f,  f3 
and  U3  are 

VI  __ L  0 

V6  V6  U 

J-  V5  0  . 

V6  V6  U 

0  0  1 

The  columns  of  U 3  form  an  orthonormal  eigenbasis  for  the  partial  frame  operator  F,  F] 
with  corresponding  eigenvalues  {/l3;i,  /t3;2,  /t3;3}  =  {|,|,0}.  For  then  =  3  iteration,  we  pick 
V3  =  I  and  cancel  the  common  terms  in 

n  3  4 

d„;3  0  | 

3  ,  ^  5 

/l«; 2  3  3 

3  5  5 

3  3 

to  obtain  J3  =  {2, 3}  and  2T3  =  {1, 3},  implying 

Pi  =  d3;3  =0,  72  =  d4;3  = 

^1  =  -^3;2  =  5,  7l  =  ^4;1  =  f- 

In  Step  B.3,  we  then  compute  the  F3  x  1  =  2  x  1  vectors  v3  and  w3  in  a  manner  analogous 
to  (3.31),  (3.32),  (3.33)  and  (3.34): 

j_l  [  Vs 

V6  3 

V3  =  ,  W3  = 

_\/5  2 

V6  J  L  3 

Note  that  in  Step  B.4,  the  role  of  permutation  matrix  11]  is  that  it  maps  the  entries  of 
v3  onto  the  1 3  indices,  meaning  that  v4  lies  in  the  span  of  the  corresponding  eigenvectors 
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In  a  similar  fashion,  the  purpose  of  the  permutation  matrices  in  Step  B.5  is  to  embed  the 
entries  of  the  2x2  matrix  W3  into  the  I3  =  {2, 3}  rows  and  J~3  =  {1, 3}  columns  of  a  3  x  3 
matrix: 
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For  the  last  iteration  n  =  4,  we  again  choose  V4  =  I  in  Step  B.l.  For  Step  B.2,  note  that 
since 

77  4  5 


A  ^  —  — 

An< 3  3  3 

J  ,  5  5 

/l”-2  3  3 

i  5  5 

3  3 

we  have  J4  =  {3}  and  274  =  {1},  implying 

0  0  1  10  0 

nJ4  =  1  0  0  ,  %4  =  0  1  0 

0  1  0  0  0  1 

Working  through  Steps  B.3,  B.4  and  B.5  yields  the  UNTF: 

_  i  t 
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We  emphasize  that  the  UNTF  F  given  in  (3.35)  was  based  on  the  particular  choice  of  outer 
eigensteps  given  in  (3.28),  which  arose  by  choosing  (x,y)  =  (0,  |)  in  (3.26).  Choosing  other 
pairs  ( x,y )  from  the  parameter  set  depicted  in  Figure  3.1(b)  yields  other  UNTFs.  Indeed, 
since  the  outer  eigensteps  of  a  given  F  are  equal  to  those  of  U F  for  any  unitary  operator 
U,  we  have  in  fact  that  each  distinct  (x,  y)  yields  a  UNTF  which  is  not  unitarily  equivalent 
to  any  of  the  others.  For  example,  by  following  the  algorithm  of  Theorem  7  and  choosing 


U i=I  and  Vn  =  I  in  each  iteration,  we  obtain  the  following  four  additional  UNTFs,  each 
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corresponding  to  a  distinct  comer  point  from  Figure  3.1(b)  of  the  parameter  set: 

1  I  °  4  4 

0  f  0  f  f  for  (*,).)  =  (i,±), 

0  0  1  Is  Hi. 

I  I  _1  1 

3  3  3  V3 

^  3^2  _3^2  TI  for  (x,  j)  =  (3,  3), 

0  I  I  V 

1  0  0  i 

F=  0  1  l  -l  -l 

0  0  f  f  # 

I  _J_  i  _I 

3  V3  3  3 

f  f  ^2  -^2  for  (x,_y)  =  (0,  |). 

0  0  I  i 

Notice  that,  of  the  four  UNTFs  above,  the  second  and  fourth  are  actually  the  same  up  to 
a  permutation  of  the  frame  elements.  This  is  an  artifact  of  our  method  of  construction, 
namely,  that  our  choices  for  outer  eigensteps,  U 1,  and  \Vn}^~'  determine  the  sequence  of 
frame  elements.  As  such,  we  can  recover  all  permutations  of  a  given  frame  by  modifying 
these  choices. 

We  emphasize  that  these  four  UNTFs  along  with  that  of  (3.35)  are  but  five  examples 
from  the  continuum  of  all  such  frames.  Indeed,  keeping  x  and  y  as  variables  in  (3.26) 
and  applying  the  algorithm  of  Theorem  7 — again  choosing  U\  -  I  and  Vn  =  I  in  each 
iteration  for  the  sake  of  simplicity — yields  the  frame  elements  given  in  Table  3.1.  Here, 
we  restrict  (x,y)  so  as  to  not  lie  on  the  boundary  of  the  parameter  set  of  Figure  3.1(b). 
This  restriction  simplifies  the  analysis,  as  it  prevents  all  unnecessary  repetitions  of  values 


for  (x,y)  =  (|,  1), 
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in  neighboring  columns  in  (3.26).  Table  3.1  gives  an  explicit  parametrization  for  a  two- 
dimensional  manifold  that  lies  within  the  set  of  all  UNTFs  consisting  of  five  elements 
in  three-dimensional  space.  By  Theorem  7,  this  can  be  generalized  so  as  to  yield  all 
such  frames,  provided  we  both  (i)  further  consider  (x,  y)  that  lie  on  each  of  the  five  line 
segments  that  constitute  the  boundary  of  the  parameter  set  and  (ii)  throughout  generalize 
Vn  to  an  arbitrary  block-diagonal  unitary  matrix,  where  the  sizes  of  the  blocks  are  chosen 
in  accordance  with  Step  B.l. 

Having  discussed  the  utility  of  Theorem  7,  we  turn  to  its  proof. 

Proof  of  Theorem  7.  (<=)  Let  {Am}^=l  and  {pn}^=l  be  arbitrary  nonnegative  nonincreasing 
sequences  and  take  an  arbitrary  sequence  of  outer  eigensteps  {{An:m}^=l}^=0  in  accordance 
with  Definition  1.  Note  here  we  do  not  assume  that  such  a  sequence  of  outer  eigensteps 
actually  exists  for  this  particular  choice  of  {Amj^=l  and  {//„}^=1;  if  one  does  not,  then  this 
direction  of  the  result  is  vacuously  true. 

We  claim  that  any  F  -  {/„}^=1  constructed  according  to  Step  B  has  the  property  that  for 
all  n  =  1  the  spectrum  of  the  frame  operator  FnF*  of  F„  =  =1  is  {/l„;m}“=| ,  and 

that  the  columns  of  U„  form  an  orthonormal  eigenbasis  for  FnF*.  Note  that  by  Lemma  3, 
proving  this  claim  will  yield  our  stated  result  that  the  spectrum  of  FF*  is  {Am}M=l  and 
that  ||/„|| 2  =  jit!  for  all  77  =  1  Since  Step  B  is  an  iterative  algorithm,  we  prove 

this  claim  by  induction  on  n.  To  be  precise,  Step  B  begins  by  letting  Uy  =  {uy;m}^=l  and 
fi  -  yfiTyU i;i .  The  columns  of  Uy  form  an  orthonormal  eigenbasis  for  F{F*  since  Uy  is 
unitary  by  assumption  and 

*  HyUy.y  777  =  1, 

FyFlUy,m  =  {Uy  ;m,/l)/l  =  Wy-m,  y/pUy-y)  y/JTyUy-y  =  fiy  (Uy-m,  Uy-y  )Uy-y  =  < 

|  0  777  #  1, 

for  all  777  =  1  ,M.  As  such,  the  spectrum  of  F{F\  consists  of /q  and  M—  1  repetitions  of 

0.  To  see  that  this  spectrum  matches  the  values  of  {Ti ;m}^f=1,  note  that  by  Definition  1,  we 
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Table  3.1:  A  continuum  of  UNTFs.  To  be  precise,  for  each  choice  of  (x,y)  that  lies  in  the 
interior  of  the  parameter  set  depicted  in  Figure  3.1(b),  these  five  elements  form  a  UNTF  for 
M3.  meaning  that  its  3  x  5  synthesis  matrix  F  has  both  unit  norm  columns  and  orthogonal 
rows  of  constant  squared  norm  |.  These  frames  were  produced  by  applying  the  algorithm 
of  Theorem  7  to  the  sequence  of  outer  eigensteps  given  in  (3.26),  choosing  U\  =  I  and 
Vn  =  I  for  all  77.  These  formulas  give  an  explicit  parametrization  for  a  two-dimensional 
manifold  that  lies  within  the  set  of  all  3  x  5  UNTFs.  By  Theorem  7,  every  such  UNTF 
arises  in  this  manner,  with  the  understanding  that  (x,  y)  may  indeed  be  chosen  from  the 
boundary  of  the  parameter  set  and  that  the  initial  eigenbasis  U\  and  the  block-diagonal 
unitary  matrices  Vn  are  not  necessarily  the  identity. 
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y(4-3AX5-3yXy-A)(2+3A-3y) 
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2  y3(2-3A)v(2-y) 


yA(3y-l)Ct-AX2+3A-3v) 
4  y3(2-3A)(l-y) 
yAy(3y-l)ly-AX2+3A-3v) 
4y3(2-3AXl-yX2-y) 


yA(3y-lXv-A)(2+3A-3y) 
4  y3(2-3A)(l-y) 
yiy(3y-l)Cy-AX2+3A-3y) 
4  -\j3(2-3x)(l—y)(2-y) 


yA(5-3yX2-A-yX4-3A--3v) 
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know  {Ti;m}^=1  interlaces  on  the  trivial  sequence  {/l0;m}^=l  =  {0}3f=l  in  the  sense  of  (3.3), 
implying  Ti;m  =  0  for  all  m  >  2;  this  in  hand,  note  this  definition  further  gives  that 
Ti;i  =  2m=i  ^i;m  =  Ah-  Thus,  our  claim  indeed  holds  for  n  =  1. 

We  now  proceed  by  induction,  assuming  that  for  any  given  77  =  1, . . . ,  A  - 1  the  process 
of  Step  B  has  produced  T7,,  =  =1  such  that  the  spectrum  of  is  {/l„;m(“=1  and 

that  the  columns  of  C/„  form  an  orthonormal  eigenbasis  for  FnF*n.  In  particular,  we  have 
FnF*Un  =  £/„D„  where  D„  is  the  diagonal  matrix  whose  diagonal  entries  are  {/f„;m}^=| . 
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Defining  Dn+\  analogously  from  {An+ we  show  that  constructing  /„+ 1  and  Un+l 
according  to  Step  B  implies  Fn+lF*+lUn+l  =  Un+lDn+1  where  Un+ 1  is  unitary;  doing  such 
proves  our  claim. 

To  do  so,  pick  any  unitary  matrix  Vn  according  to  Step  B.l.  To  be  precise,  let  Kn 
denote  the  number  of  distinct  values  in  {An;m}^=l,  and  for  any  k  =  1, . . . ,  Kn,  let  Lirk  denote 
the  multiplicity  of  the  kth  value.  We  write  the  index  m  as  an  increasing  function  of  k  and 
/,  that  is,  we  write  {An;m}^=1  as  {A n-,m(k,i)}fh  f"\k  where  m(k,  l )  <  m(k\  /')  if  k  <  k'  or  if  k  =  k' 
and  /  <  /'.  We  let  Vn  be  an  M  x  M  block-diagonal  unitary  matrix  consisting  of  K  diagonal 
blocks,  where  for  any  k  =  1, . . . ,  K,  the  kth  block  is  an  Ln;k  x  L,rk  unitary  matrix.  In  the 
extreme  case  where  all  the  values  of  {d„;m}“=1  are  distinct,  we  have  that  V„  is  a  diagonal 
unitary  matrix,  meaning  it  is  a  diagonal  matrix  whose  diagonal  entries  are  unimodular. 
Even  in  this  case,  there  is  some  freedom  in  how  to  choose  Vn;  this  is  the  only  freedom  that 
the  Step  B  process  provides  when  determining  fn+\.  In  any  case,  the  crucial  fact  about  V„ 
is  that  its  blocks  match  those  corresponding  to  distinct  multiples  of  the  identity  that  appear 
along  the  diagonal  of  Dn,  implying  DnVn  -  VnDn. 

Having  chosen  Vn,  we  proceed  to  Step  B.2.  Here,  we  produce  subsets  In  and  of 
{1, . . .  ,M)  that  are  the  remnants  of  the  indices  of  {An-m}^I=l  and  {An+ i;m}^f=1,  respectively, 
obtained  by  canceling  the  values  that  are  common  to  both  sequences,  working  backwards 
from  index  M  to  index  1.  An  explicit  algorithm  for  doing  so  is  given  in  Table  3.2.  Note 
that  for  each  m  =  M, ,  1  (Line  03),  we  either  remove  a  single  element  from  both  and 

(Lines  04-06)  or  remove  nothing  from  both  (Lines  07-09),  meaning  that  In  :=  1 

and^T),  :=  SFn^  have  the  same  cardinality,  which  we  denote  R„.  Moreover,  since  {An+ i;m}^f=1 
interlaces  on  {/l„;m)“=| ,  then  for  any  real  scalar  A  whose  multiplicity  as  a  value  of  {An;m}^=1 
is  L,  we  have  that  its  multiplicity  as  a  value  of  {An+ i;m}ff=1  is  either  L  -  l,LorL  +  1.  When 
these  two  multiplicities  are  equal,  this  algorithm  completely  removes  the  corresponding 
indices  from  both  In  and  27».  On  the  other  hand,  if  the  new  multiplicity  is  L  -  1  or  L  +  1, 
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then  the  least  such  index  in  J„  or  J~n  is  left  behind,  respectively,  leading  to  the  definitions 
of  In  or  J„  given  in  Step  B.2.  Having  these  sets,  it  is  trivial  to  find  the  corresponding 
permutations  7 rj„  and  njn  on  {1, . . . ,  M)  and  to  construct  the  associated  projection  matrices 
nr  and  nT  . 

n  n 

D 

We  now  proceed  to  Step  B.3.  For  the  sake  of  notational  simplicity,  let  {/3r}r "x 
and  [yr)Rrlx  denote  the  values  of  {An;m}meIn  and  {An+l;m}meJn,  respectively.  That  is,  let 
j3„Tn(,„)  =  An;m  for  all  m  e  I „  and  ynjn(m)  -  An+i;m  for  all  m  e  J~n.  Note  that  due  to  the 
way  in  which  In  and  were  defined,  we  have  that  the  values  of  and  {yr}R"t  are 

all  distinct,  both  within  each  sequence  and  across  the  two  sequences.  Moreover,  since 
and  {An+ i-m}mejn  are  nonincreasing  while  nIn  and  nJn  are  increasing  on  In  and 
Jn  respectively,  then  the  values  and  \yr\R’lx  are  strictly  decreasing. 


D  D 

We  further  claim  that  {yr}r"j  interlaces  on  {f3r}rlv  To  see  this,  consider  the  four 
polynomials: 


M 

Pfl^X)  ~  |  A„-J„  ), 

m=  1 
Rn 

b(x)  =  ]~”[(x-/3,.), 

r=  1 


M 

Pn+l(.X)  ~  J 

m=  1 
Rn 

c(x)  =  r>  -7r).  (3.36) 

r=  1 


Since  and  {yr}^"1  were  obtained  by  canceling  common  terms  from  {T„;m}^=1  and 

we  have  that  pn+i(x)/p„(x)  =  c(x)/b(x )  for  all  *  g  {T„;m}^=1.  Writing  any 
r  =  1,  as  r  =  nIn(m)  for  some  m  e  J„,  we  have  that  since  {T„;m}“=1  E  {An+  i;m}^=1, 

applying  the  “only  if’  direction  of  Lemma  5  with  “p(x)”  and  “g(x)”  being  p„(x)  and  p„+1(x) 
gives 


r  /  r  ,  ,  .Ai+l(x) 

hm(x  T  =  llm  (*  -  ^«;m) - y—  <  0. 

X—*flr  »(x)  pn(x ) 


(3.37) 


Since  (3.37)  holds  for  all  r  =  1, . . .  ,Rn,  applying  “if’  direction  of  Lemma  5  with  “p(x)” 
and  “g(x)”  being  /?(x)  and  c(x)  gives  that  {yr}^"1  indeed  interlaces  on 

Taken  together,  the  facts  that  {fi,)R'i\  and  {y,}f",  are  distinct,  strictly  decreasing,  and 
interlacing  sequences  implies  that  the  i?„xl  vectors  vn  and  w„  are  well-defined.  To  be 
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Table  3.2:  An  explicit  algorithm  for  computing  the  index  sets  In  and  J~n  in  Step  B.2  of 
Theorem  7. 


01 

I\M) 

,M] 

02 

JT  :={!,.• 

-,M] 

03 

for  m  =  M, . . . 

.,1 

04 

if  n\m  ^ 

05 

n 

I{n]  \  {m} 

06 

<j(m- 1) 

-  Jim)  \  ! m'  1  where  m' 

07 

else 

08 

j-(m-l) 

n  ■ 

f(m) 

72 

09 

q-(m-l) 

n  • — 

■  o'  72 

10 

end  if 

11 

end  for 

12 

I  —  J(1) 

n  •  n 

13 

T  •=  T(1) 

Am)  . 


precise,  Step  B.3  may  be  rewritten  as  finding  vn(r),wn(r')  >  0  for  all  r,r'  =  1 . . . ,  R„  such 
that 

R„  R„ 

[”[  (fir  ~  Jr"  )  Y\  (yr'  ~  Pr"  ^ 

Mr)]2  =  -  ^ - ,  [Mr')]2  =  ^ - •  (3.38) 

Y\Vr-M  P]  (Jr'  ~  Yr") 

r"= 1  r"= 1 

r"+r  r"±r' 

Note  the  fact  that  the  /?,.’s  and  yr’s  are  distinct  implies  that  the  denominators  in  (3.38)  are 

D 

nonzero,  and  moreover  that  the  quotients  themselves  are  nonzero.  In  fact,  since  {pr)r=x  is 
strictly  decreasing,  then  for  any  fixed  r,  the  values  {/ 3r  -  /3 +r  can  be  decomposed  into 
r  -  1  negative  values  \[3r  -  pr”Yr7'=\  and  Rn  -  r  positive  values  {J3r  -  fir»}^,n,=r+l.  Moreover, 

D  D  D 

since  {/3,-}r=i  E  {yr}rlv  then  for  any  such  r,  the  values  {/3r  -  7r"}r"=i  can  he  broken  into  r 
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negative  values  {fir  -  y,"Vr„=l  and  R„  -  r  positive  values  {fir  -  y,"}f/,'=r+1 .  With  the  inclusion 
of  an  additional  negative  sign,  we  see  that  the  quantity  defining  [v„(r)]2  in  (3.38)  is  indeed 
positive.  Meanwhile,  the  quantity  defining  [wn(r'  j\2  has  exactly  r'  —  1  negative  values  in 
both  the  numerator  and  denominator,  namely  {yr>  -  fir»  and  \yr>  -  yr"  ,  respectively. 

Having  shown  that  the  v„  and  wn  of  Step  B.3  are  well-defined,  we  now  take  fn+l 
and  Un+ 1  as  defined  in  Steps  B.4  and  B.5.  Recall  that  what  remains  to  be  shown  in  this 
direction  of  the  proof  is  that  Un+ 1  is  a  unitary  matrix  and  that  Fn+ i  =  satisfies 

F n+ 1 F*n+ 1 Un+ 1  =  Un+1Dn+v  To  do  so,  consider  the  definition  of  Un+  i  and  recall  that  Un  is 
unitary  by  the  inductive  hypothesis,  Vn  is  unitary  by  construction,  and  that  the  permutation 
matrices  11/  and  n jn  are  orthogonal,  that  is,  unitary  and  real.  As  such,  to  show  that  Un+\ 
is  unitary,  it  suffices  to  show  that  the  Rn  x  Rn  real  matrix  Wn  is  orthogonal.  To  do  this, 
recall  that  eigenvectors  corresponding  to  distinct  eigenvalues  of  self-adjoint  operators  are 
necessarily  orthogonal.  As  such,  to  show  that  Wn  is  orthogonal,  it  suffices  to  show  that  the 
columns  of  Wn  are  eigenvectors  of  a  real  symmetric  operator.  To  this  end,  we  claim 

(Dn.In  +  vnvl)Wn  =  WnDn+UJn,  Wl Wn(r,  r)  =  1,  Vr  =  1, . . . ,  R,„  (3.39) 


where  DirIn  and  Dn+ l;jn  are  the  Rn  x  R„  diagonal  matrices  whose  rth  diagonal  entries  are 
given  by  [i,  =  An.n- i(r)  and  y,  =  An+l.n- i(r),  respectively.  To  prove  (3.39),  note  that  for  any 

’  Fn v  ’  SFn v 

r  yf  —  I  /? 


mrJn  +  vnvTn)Wn](r,  /)  =  (. Dn.JnWn)(r ,  /)  +  (vnvTnWn)(r,  /) 


=  /3,  Wn(r,  r  )  +  v„(r)  ^  vn(r")Wn(r" ,  /).  (3.40) 


r"= 1 


Rewriting  the  definition  of  W„  from  Step  B.5  in  terms  of  {/?,-}, ~i  and  gives 

vn(r)wn(r') 


Wn(r,r')  = 


Jf  ~/3r 


(3.41) 


46 


Substituting  (3.41)  into  (3.40)  gives 


[(Dn,Iit+vnvTn)Wn\(r,r')=fS, 


Vn(r)wn(r') 
Jr’  ~  fir 


+  vn(r)  ^  v„(r") 


Vn(r")wn(r') 


=  vn(r)wn(r') 


fir 


r"=  1 

R„ 


Jr’  -  fir 


+ 


ZL  Vn 

7  , 


r"= 1 


TV  fir 

[Vnir")]2' 

Jr'  -fir" 


(3.42) 


Simplifying  (3.42)  requires  a  polynomial  identity.  Note  that  the  difference  Y\r"=l(x-yr")  - 
YlXfix  -  fir")  of  two  monic  polynomials  is  itself  a  polynomial  of  degree  at  most  Rn  -  1, 
and  as  such  it  can  be  written  as  the  Lagrange  interpolating  polynomial  determined  by  the 
Rn  distinct  points  {fir}Rr'l{- 

Rn  Rn  Rn  ,  Rn  .  Rn 


f](*  -  Jr")  -  Y[{x~fir»)  =  ^(|  \(fir"  ~  Jr)  ~  o) 


r"= 1 


r"= 1  r=  1 
R, 
Rn 


r 

r+R 


(X~  fir) 
f  (fir"  -fir) 


JV/7 

Y\(fir"-yr) 


— r>-«- 


(3.43) 


r"= 1 


{\{fir"  -fir)  r*r" 


r=  1 
r-f-r" 


Recalling  the  expression  for  [vn(r)]2  given  in  (3.38),  (3.43)  can  be  rewritten  as 

Rn  Rn  Rn  R„ 

Y\(x-fir")  -  ]~[(.x;  -  y  r")  =  ^[v„(r")]2  Y [(*  -fir). 


(3.44) 


r"= 1 


r"= 1 


r"=  1 


r=l 

r-f-r" 


iy-n 

Dividing  both  sides  of  (3.44)  by  |~ |(.r  -A")  gives 


r"=l 


"U2 


i  _  n  ^  ~ ^")  -  v  Mr")] 

1  1  ~  2-i  (x- 


Vjc  £  (A 


r’r=l  • 


(3.45) 


v=l  (X  _  fir")  (-*  -  A") 

For  any  /  =  1, . . .  ,  letting  jc  =  in  (3.45)  makes  the  left-hand  product  vanish,  yielding 
the  identity: 

r  V  M^")]2  vy  '  i  d  s?  a ^ 

1  =  2j  7~, — ~  Vr  =  1, . . .  ,Rn.  (3.46) 


^  M  -  A") 
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Substituting  (3.46)  into  (3.42)  and  then  recalling  (3.41)  gives 


+  vnvl)Wn](r,  r')  =  vn{r)wn{r')  (  —  +  l) 

Vyr'-Pr  / 

v„(r)w„(r/) 

=  % - 

7r'  ~  fir 

=  yr'W„(r,r') 

=  (W„Z)„+i^„)(r,r').  (3.47) 


As  (3.47)  holds  for  all  r,r'  =  l,...,/?„  we  have  the  first  half  of  our  claim  (3.39).  In 
particular,  we  know  that  the  columns  of  Wn  are  eigenvectors  of  the  real  symmetric  operator 
D„.jn  +  vnvjj  which  correspond  to  the  distinct  eigenvalues  { y,-)Rrn=l ■  As  such,  the  columns  of 
Wn  are  orthogonal.  To  show  that  Wn  is  an  orthogonal  matrix,  we  must  further  show  that 
the  columns  of  Wn  have  unit  norm,  namely  the  second  half  of  (3.39).  To  prove  this,  at  any 
x  i  we  differentiate  both  sides  of  (3.45)  with  respect  to  x  to  obtain 

R„  R„  ,  , .  ^  R„ 


Jr"  —  fir 


ZL  VnK 

(  Y  — 


-  [Vn{r")f 


iyn  r  lxn  /  \ 

V  [fl  x  Z  y',') 

h  I  U  -  Pr)  J  (X  ~  fir^Y  (X  -  fi,„) 


Vjc  t  [fi 


r>r=  r 


(3.48) 


r=£r 


For  any  r'  =  1  letting  .r  =  yr>  in  (3.48)  makes  the  left-hand  summands  where 

r"  r'  vanish;  by  (3.38),  the  remaining  summand  where  r"  -  r'  can  be  written  as: 

R 


|  ~[(7r>  ~  Jr ) 


1 


r=  1 
r+r’ 


[wn{r')Y 


]  J (y r'  -fir) 


\r\— 

U  1  (y. 


(7r’  -  yr ) 


yr'  -  fir 


r±r' 


l  ijr'  ~  fir)  Hjr'  ~  fir'Y  ^  (jr'  -fir") 


(v./ 


[v„(r")] 


"m2 


^2• 


(3.49) 


r=  1 
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We  now  use  this  identity  to  show  that  the  columns  of  Wn  have  unit  norm;  for  any 
/  =  !,..  .,Rn,  (3.41)  and  (3.49)  give 


{WlWn)(r\r')  =  ^JWn(r",r')f 

r"= 1 

_  y  (Vn(r")wn(/) 

^,4,'  yr'-Pr" 


=  \wn(r')Y 


r  ,  ,v2  [vn(r")f 

[w'(r)1 

1 


[w„(r')]2 


Having  shown  that  W„  is  orthogonal,  we  have  that  Un+ 1  is  unitary.  For  this  direction  of  the 
proof,  all  that  remains  to  be  shown  is  that  Fn+lF*+lUn+l  =  Un+lDn+1.  To  do  this,  we  write 
Fn+iFn+i  =  F„Fn  +  /,)+i.Ci  and  recall  the  definition  of  Un+l: 


Fn+lFn+lU„+l  =  (FnK  +  fn+lfn+l)UnVnUTI„  ^  ^ 

0  I 


-  F  F*TI  V  TTt 

_  rnrnun  vnL1I„ 


wn 

0 

nJn  +  fn+lfZ+1unvnnl 

w„ 

0 

0 

I 

0 

i 

n, y.  (3.50) 


To  simplify  the  first  term  in  (3.50),  recall  that  the  inductive  hypothesis  gives  FnF*Un 
UnDn  and  that  V„  was  constructed  to  satisfy  DnVn  =  VnDn,  implying 


FnF:UnVnIll 


Wn  0  ^  Wn  0 

n =  unvnon  ul  uJn 

0  1  0  1 


UnVjFrQlrDjFr) 


0 


II/,-  (3.51) 


0  I 
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To  continue  simplifying  (3.51),  note  that  IIr  D  III  is  itself  a  diagonal  matrix:  for  any 
m,  m'  =  1, . . . ,  M,  the  definition  of  the  permutation  matrix  fljn  given  in  Step  B.2  gives 


(13 T  Dn nj  )(m,  m)  =  (Dn IlJ  8W>,  U]  6m) 

I  /^n;;r“1(m)>  ^  Ht  ^ 

J-n  I  • 

0,  m  4  m  . 

That  is,  nr  D  fll  is  the  diagonal  matrix  whose  first  Rn  diagonal  entries  = 

n  -L  n  F  i 

D 

{An.n-i,r)}  \  match  those  of  the  aforementioned  Rn  x  R„  diagonal  matrix  Dn  In  and  whose 
remaining  M  - Rn  diagonal  entries  \AIV7I  i  (m)}^=R  +1  form  the  diagonal  of  an  (M  -  Rn)  x  (M  - 

’In  n 

Rn )  diagonal  matrix  D„.jc : 


Uj  D(,n] 

1  n  n  1  n 


DnJn  0 
0  Dn:Ic„ 


(3.52) 


Substituting  (3.52)  into  (3.51)  gives 


F  F*IJ  V  TT1 

r  nr  nun  %li/„ 


wn  0 

D„jn  0 

Wn  0 

%,  =  UnVnU}n 

0  I 

0  A5;jc 

0  I 

n 


Unvnul 


Dlvjn  Wn  0 
0  Dn-n 


n j. 

n 


(3.53) 


To  simplify  the  second  term  in  (3.50),  we  recall  the  definition  of  fn+ 1  from  Step  B.4: 

fn+if:+iUnvnu}\  in 


o  i 


-J.  unvn  m 


o 


i>T  o 


w„  0 


0  I 


n 


Jn 


Unvn  ul 


v  vlW  0 

ynynTTn  w 


o  o 


n 


Jn 


(3.54) 


50 


Substituting  (3.53)  and  (3.54)  into  (3.50),  simplifying  the  result,  and  recalling  (3.39)  gives 


Fn+lF*n+1un+l  =  unvn  nj 


(D„,In  +  v,yn)Wn  0 

0  Dnjc 


n 


Jn 


=  unvnnl 


WnD„+ u,  0 

0  Dnjcn 


Uj. 

n 


By  introducing  an  extra  permutation  matrix  and  its  inverse  and  recalling  the  definition  of 
Un+ 1,  this  simplifies  to 


wn 

0 

Dn+ 1  \jn 

0 

Fn+lF*+lUn+l  =  unvH  njn 

nT  n1, 

•J  n  n 

0 

I 

0 

D>rJcn 

-  Un+1Uj 

n 


D 


n+l-Jn 


0 


D„ 


n-J\ 


n 


Jn' 


(3.55) 


We  now  partition  the  [An+  i;m}^=1  of  D„+1  into  J~n  and  and  mimic  the  derivation  of  (3.52), 
writing  Dn+i  in  terms  of  Dn+l.jn  and  Dn+ 1 .  Note  here  that  by  the  manner  in  which  In  and 
Jn  were  constructed,  the  values  of  {A„;m}m€jc  are  equal  to  those  of  {An+i;m}jc,  as  the  two 
sets  represent  exactly  those  values  which  are  common  to  both  j A,rjn\^i= ,  and  \All+]-m}^l=r  As 
these  two  sequences  are  also  both  in  nonincreasing  order,  we  have  Dn.jc  =  Dn+ ,  :jc  and  so 


iAn+ 1  n  Tn  - 


Dn+lJn 

0 

Dn+\;Jn 

0 

0 

Dn+\:JC„ 

0 

Dn;Icn 

(3.56) 


Substituting  (3.56)  into  (3.55)  yields  Fn+lF*+lUn+l  =  Un+lDn+l,  completing  this  direction 
of  the  proof. 


(=>)  Let  { /{,„ }^= ,  and  {//„}^=1  be  any  nonnegative  nonincreasing  sequences,  and  let 
F  =  {fn)n=i  be  any  sequence  of  vectors  whose  frame  operator  FF*  has  {Am}^=l  as  its 
spectrum  and  has  ||y;;||2  =  /un  for  all  n  =  1  We  will  show  that  this  F  can  be 

constructed  by  following  Step  A  and  Step  B  of  this  result.  To  see  this,  for  any  n  =  1, . . . ,  N, 
let  Fn  =  =1  and  let  {An;m}ffI=l  be  the  spectrum  of  the  corresponding  frame  operator 
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FnF*.  Letting  A()m  :=  0  for  all  m,  the  proof  of  Theorem  2  demonstrated  that  the  sequence 
of  spectra  {{An:m}^=l}^=0  necessarily  forms  a  sequence  of  outer  eigensteps  as  specified  by 
Definition  1.  This  particular  set  of  outer  eigensteps  is  the  one  we  choose  in  Step  A. 

All  that  remains  to  be  shown  is  that  we  can  produce  our  specific  F  by  using  Step  B. 
Here,  we  must  carefully  exploit  our  freedom  to  pick  Lj  and  the  V„  s;  the  proper  choice  of 
these  unitary  matrices  will  result  in  F,  while  other  choices  will  produce  other  sequences 
of  vectors  that  are  only  related  to  F  through  a  potentially  complicated  series  of  rotations. 
Indeed,  note  that  since  {{T„;m}£f=1}^=0  is  a  valid  sequence  of  outer  eigensteps,  then  the  other 
direction  of  this  proof,  as  given  earlier,  implies  that  any  choice  of  U\  and  V„’s  will  result  in 
a  sequence  of  vectors  whose  outer  eigensteps  match  those  of  F.  Moreover,  quantities  that 
we  considered  in  the  other  direction  of  the  proof  that  only  depended  on  the  choice  of  outer 
eigensteps,  such  as  I n,  J~n,  {/?r}f=j,  ijr) etc.,  are  thus  also  well-defined  in  this  direction; 
in  the  following  arguments,  we  recall  several  such  quantities  and  make  further  use  of  their 
previously-derived  properties. 

To  be  precise,  let  U\  be  any  one  of  the  infinite  number  of  unitary  matrices  whose  first 
column  Mi;i  satisfies  f\  =  \fjT\ u\  \.  We  now  proceed  by  induction,  assuming  that  for  any 
given  n  =  1, . . . ,  A  -  1,  we  have  followed  Step  B  and  have  made  appropriate  choices  for 
{ Vn>  ("rj,  so  as  to  correctly  produce  Fn  =  {/„/(”, =1 ;  we  show  how  the  appropriate  choice  of 
V„  will  correctly  produce  /„+  \ .  To  do  so,  we  again  write  the  nth  spectrum  {An;m}^=1  in  terms 
of  its  multiplicities  as  {An;m(k,i)}f=  j  For  any  k  =  1, . . .  ,Kn,  Step  B  of  Theorem  2  gives 
that  the  norm  of  the  projection  of  fn+ 1  onto  the  kth  eigenspace  of  FnF*  is  necessarily  given 
by 


11^ 


n:A, 


■n\m(k,  1) 


f. 


n+lll  — 


r  /  3  ,Pn+l(x) 

lim  (x  -  - — —  , 

X  *Antm(k,l)  Pn\X) 


(3.57) 


where  pn(x)  and  pn+ 1 (x)  are  defined  by  (3.36).  Note  that  by  picking  /  =  1,  A,rj„(kA) 
represents  the  first  appearance  of  that  particular  value  in  {An;m}^=1.  As  such,  these  indices 
are  the  only  ones  that  are  eligible  to  be  members  of  the  set  In  found  in  Step  B.2.  That 
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is,  In  c  {m(k,  1)  :  k  =  1, . . .  ,K„}.  However,  these  two  sets  of  indices  are  not  necessarily 
equal,  since  Im  only  contains  m’s  of  the  form  m(k,  1)  that  satisfy  the  additional  property 
that  the  multiplicity  of  An;m  as  a  value  in  exceeds  its  multiplicity  as  a  value  in 

Un+u,„C=r  To  be  precise,  for  any  given  k  =  1  ,...,Kn,  if  m(k,  1)  G  Icn  then  An.m(U)  appears 
as  a  root  of  pn+  i(x)  at  least  as  many  times  as  it  appears  as  a  root  of  pn(x),  meaning  in  this 
case  that  the  limit  in  (3.57)  is  necessarily  zero.  If,  on  the  other  hand,  m(k,  1)  G  In,  then 
writing  nIn(m(k,  1))  as  some  r  G  {1, . . .  and  recalling  the  definitions  of  b(x)  and  c(x) 
in  (3.36)  and  v(r)  in  (3.38),  we  can  rewrite  (3.57)  as 


WPnpJn+lW2  =-\im(x-fir)P'"'iX)  =-lim  ix-pr)^- 
x^J3r  Pn{x)  x-*Pr  b(x) 


n-n 

f[  (Pr-7r») 

^ - =[v„(r)]2.  (3.58) 

Y\(fir~Pr") 

r"= 1 
r"+r 


As  such,  we  can  write  f,+l  as 


fn+ 1  —  Pn-,A„Mk,i)fn+\ 
k=  1 

Rn 

—  ^  ^  I f n+\ 

r=  1 

Rn  | 

—  ^  Vnil)  ~T\Pn:prfn+\ 

v„(r) 

=  X  Vn(/Tj„(m))  Vn(n^(m))  P "An Jn (m)fn+  I 


inel,, 


(3.59) 


where  each  v  {m))Pn:i3ni  has  unit  norm  by  (3.58).  We  now  pick  a  new  orthonormal 
eigenbasis  U„  :=  {un;m}^=l  for  FnF*  that  has  the  property  that  for  any  k  -  1, . . . ,  K„.  both 
{un;m(k,i)}fl'i  and  {un-m(k,i)}f=i  sPan  the  same  eigenspace  and,  for  every  m(k,  1)  G  J„,  has  the 
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additional  property  that  wj;;m(jU)  =  ViilIIi  Jm(U))) Pnfi„In „./«+ 1  ■  As  such,  (3-59)  becomes 


fn+i  =  ^  vn(nIn(m))un;m 

meln 

=  un  Yj  Vn(^iJm))Sm 
meln 
R„ 

=  U„  ^  Vn{r)6n- I(r) 

r=l 


R„ 


#„nj_  X 


4,nt 


r=  1 

V„ 

0 


(3.60) 


Letting  V„  be  the  unitary  matrix  V„  =  U*Un,  the  eigenspace  spanning  condition  gives  that 
Vn  is  block-diagonal  whose  kth  diagonal  block  is  of  size  Ln^  x  L„./(.  Moreover,  with  this 
choice  of  Vn,  (3.60)  becomes 


Ui  =  unu;unu}n 

Vn 

=  unvn  nl 

Vn 

0 

0 

meaning  that  fn+]  can  indeed  be  constructed  by  following  Step  B.  □ 
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IV.  Top  Kill  algorithm  and  the  characterization  of  the  set  of  all  eigensteps 


In  this  chapter,  we  continue  the  work  of  Chapter  3  and  show  how  to  explicitly 
construct  every  sequence  of  eigensteps — a  sequence  of  interlacing  spectra — as  required 
by  Step  A  of  Theorem  2.  The  major  results  are  Theorem  16  which  provides  the  explicit 
Top  Kill  algorithm  for  constructing  a  sequence  of  eigensteps,  and  Theorem  17  which  gives 
a  complete  characterization  of  the  set  of  all  eigensteps  for  a  given  {An}„=l  and  .  When 

these  results  are  combined  with  those  in  Chapter  3,  they  provide  a  complete  solution  to  the 
following  problem: 

Problem  9.  Given  any  nonnegative  nonincreasing  sequences  {Tm}^=|  and  {/r„)(^=|,  construct 
cdl  F  =  whose  frame  operator  FF*  has  spectrum  {dm}^=|  and  for  which  \\f,\\2  =  pn 

for  cdl  n. 

To  proceed,  we  first  recall  that  any  F  =  {fn}„={  for  which  FF*  has  {Am}^=l  as  its 
spectrum  and  for  which  ||/„|| 2  =  pn  for  all  n  generates  a  sequence  of  outer  eigensteps.  In 
Chapter  3,  Theorem  2  proves  that  the  converse  of  this  statement  is  also  true.  As  already 
noted,  Theorem  2  is  not  an  easily-implementable  algorithm.  While  we  have  already  seen 
that  Step  B  can  be  made  more  explicit  (Theorem  7),  Step  A  is  still  rather  vague.  The 
techniques  in  this  chapter  will  make  Step  A  explicit,  with  our  main  result  being: 

Theorem  10.  Let  {Amj^=l  and  {p„}^=l  he  nonnegative  and  nonincreasing  where  M  <  N. 
There  exists  a  sequence  of  vectors  F  =  |  /„}')=I  in  CM  whose  frame  operator  FF*  has 
spectrum  {Am}^=1  and  for  which  \\fn\\ 2  =  pn  for  cdl  n  if  and  only  if  {Am}%=l  U  {0}^=M+1  > 
{pn}^=].  Moreover,  if{Am}^=l  U  {0}^=M+1  >  {pn}^=],  then  every  such  F  can  be  constructed  by 
the  following  process: 

A.  Let{AN,m)Z=x  :=UmC=r 

For  n  =  N, . . . ,  2,  construct  {An-  i;m}^=1  in  terms  of{An:m}^=l  as  follows: 
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For  each  k  =  M, . . . ,  1,  ifk  >  n  -  1,  take  d„_ 1;fc  :=  0. 
Otherwise,  pick  any  An-i-k  e  where 


M 


M 


A-n-l;k  • —  H13.X  ^  ^  ^ n\m  ^  ^  l^n  (? 

m-k  m=k+ 1 

1  k 


M 


Bn-l;k  null  1 min,  I  ^  J  ^  ^  ^  ^  ^/i— l;/i 


m—l  m=l+ 1 


ra=£+l 


//ere,  we  use  the  convention  that  An-M+i  =  0,  and  that  sums  over  empty  sets  of  indices 
are  zero. 


B.  Follow  Step  B  of  Theorem  2. 

Conversely,  any  F  constructed  by  this  process  has  {Am}^=l  as  the  spectrum  of  FF*  and 
\\fi\\2  -  pn  for  all  n,  and  moreover,  FnF*  has  spectrum  {An;m}^=1. 

In  Section  4.1,  we  begin  by  discussing  how  solving  Problem  9  via  Theorem  2  suffices 
to  construct  all  Schur-Horn  matrices.  We  also  introduce  the  notion  of  inner  eigensteps. 
While  outer  eigensteps  give  the  spectra  of  the  partial  frame  operators,  inner  eigensteps  will 
give  the  spectra  of  the  partial  Gram  matrices  F*nFn.  In  Section  4.2,  we  visualize  the  inner 
eigenstep  construction  problem  in  terms  of  iteratively  building  a  staircase.  We  then  provide 
a  new  algorithm,  called  Top  Kill ,  which  produces  a  sequence  of  inner  eigensteps  whenever 
possible  to  do  so.  Finally,  in  Section  4.3,  we  find  an  explicit  parametrization  of  the  set  of 
all  valid  inner  eigensteps  for  a  given  |  An  }^= ,  and  {p„}^=1. 

4.1  Preliminaries 

In  this  section,  we  first  further  detail  the  connection  between  the  Schur-Horn  Theorem 
and  Problem  9.  We  then  reformulate  Step  A  of  Theorem  2  in  terms  of  inner  eigensteps 
which  are  an  alternative,  but  mathematically  equivalent  version  of  the  outer  eigensteps 
defined  in  Chapter  3;  it  turns  out  that  it  is  more  convenient  to  work  with  spectra  whose 
number  of  elements  equals  that  of  {pn)^=v  namely  N. 
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By  applying  the  Schur-Hom  Theorem  (page  15)  to  the  Gram  matrix  F*F,  whose 
diagonal  entries  are  given  by  {||/„||2(^=l  and  whose  spectrum  is  {An}^=1,  we  are  able  to 
determine  when  a  frame  with  prescribed  spectrum  and  lengths  exists.  Here  it’s  important  to 
note  that  the  spectrum  of  the  Gram  matrix  is  a  zero-padded  version  of  the  spectrum  {Am}^=1 
of  the  frame  operator  FF*.  The  only  difference  between  the  eigenvalues  of  the  Gram  matrix 
and  the  frame  operator  is  zero  eigenvalues;  that  is,  An;m  =  0  for  M  <  n  <  N.  Thus  the  Schur- 
Horn  Theorem  implies  that  Problem  9  is  feasible  if  and  only  if  ]/r„}^=1  is  majorized  by 
{Am}w=  i  padded  with  N  -  M  zeros.  Indeed,  if  Problem  9  has  a  solution  F,  then  G  =  F*F  has 
spectrum  {dm}^=1U{0}^=M+1  and  diagonal  {/un}^=v  and  so  {/i„}^=1  <  R»}"=IU{ 0}^=M+1  by  the 
Schur-Horn  Theorem.  Conversely,  if  {/jn  }Nn=\  <  {Am}^=  i  U  {0}^=M+1  then  the  corresponding 
Schur-Horn  matrix  G  can  be  unitarily  diagonalized: 

r  i|z)i  ol  [vr 

G  =  VDV*  =  Vi  v2  =VlD1V*1, 

L  [  0  oj  [V* 

l 

where  D{  is  an  M  x  M  diagonal  matrix  with  diagonal  {Am}M=l;  the  matrix  F  =  D\  P*  is  then 
one  solution  to  Problem  9.  This  line  of  reasoning  is  well-known  [1,  19]. 

We  follow  an  alternative  approach  that  is  modeled  in  that  of  [33]:  rather  than  use  the 
Schur-Horn  Theorem  to  determine  the  feasibility  of  Problem  9,  we  instead  independently 
find  all  solutions  to  Problem  9 — see  Theorem  10 — and  then  use  these  matrices  to  construct 
all  Schur-Hom  matrices.  To  be  precise,  note  that  though  the  Schur-Hom  Theorem  applies  to 
all  self-adjoint  matrices  G,  it  suffices  to  consider  the  case  where  G  is  positive  semide finite. 
Indeed,  any  self-adjoint  matrix  G  can  be  written  as  G  =  G  +  a l  where  G  is  positive 
semidefinite  and  a  <  AmiJG);  it  is  straightforward  to  show  that  the  spectrum  { An }^=  {  of 
G  majorizes  its  diagonal  {/)„}^=1  if  and  only  if  the  spectrum  {An}^=1  =  { A,,  -  a]^=1  of  G 
majorizes  its  diagonal  {/r„}^=1  =  \p„  -  a}Nn=\ ■  Moreover,  since  G  is  positive  semidefinite, 
it  has  a  Cholesky  factorization  G  =  F*F  where  F  6  CNxN.  Regarding  F  as  the  synthesis 
operator  of  some  sequence  of  vectors  {/„(^=|  in  C'v,  we  are  thus  reduced  to  Problem  9  in 
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the  special  case  where  M  =  N.  Presuming  for  the  moment  that  Theorem  10  is  true,  we 
summarize  the  above  discussion  as  follows: 

Theorem  11.  Given  nonincreasing  sequences  {A„}^=1  and  {fin)hn={  such  that  {An}^=l 
majorizes  {fin)^=v  every  self-adjoint  matrix  G  with  spectrum  {An}^=1  and  diagonal  {pn}„=l 
can  be  constructed  as  G  =  F*F  +  al  where  F  is  any  matrix  constructed  by  taking  any 
a  <  Amin(G)  and  applying  Theorem  10  where  An  :  =  A„  -  a  and  pn  :=/)„-  a.  Moreover,  any 
G  constructed  in  this  fashion  has  the  desired  spectrum  and  diagonal. 

The  remainder  of  this  chapter  is  focused  on  solving  Problem  9.  In  particular,  we 
focus  on  Part  A  of  Theorem  2  of  finding  a  valid  sequence  of  outer  eigensteps  for  any 
given  nonnegative  nonincreasing  sequences  {Am}%=l  and  {p„}^=v  Recall  Example  8  from 
Chapter  3.  While  solving  for  the  conditions  upon  the  eigensteps  was  fairly  simple  in 
that  example,  its  complexity  will  quickly  increase  as  M  and  N  get  larger.  Finding  all 
valid  sequences  of  eigensteps  (3.26)  often  requires  reducing  a  large  system  of  linear 
inequalities  (3.27).  The  goal  of  the  remaining  sections  of  this  chapter  is  to  derive  a 
more  efficient  algorithm  for  systematically  finding  the  necessary  conditions  upon  a  given 
sequence  of  eigensteps.  It  turns  out  that  this  method  is  more  easily  understood  in  terms 
of  an  alternative  but  equivalent  notion  of  eigensteps.  To  be  clear,  for  any  given  sequence 
of  outer  eigensteps  {{A„-m}^=l}^=0,  recall  from  Theorem  2  that  for  any  n  =  1 , ,7V,  the 
sequence  {An;m}^=1  is  the  spectrum  of  the  M  x  M  frame  operator  (3.4)  of  the  nth  partial 
sequence  Fn  -  {fn>}n,=v  In  the  theory  that  follows,  it  is  more  convenient  to  instead  work 
with  the  spectrum  {An;m}’’n=l  of  the  corresponding  n  x  n  Gram  matrix  F*Fn;  we  use  the 
same  notation  for  both  spectra  since  {An;m}"n=1  is  a  zero-padded  version  of  {An;m}^=1  or  vice 
versa,  depending  on  whether  n  >  M  or  n  <  M.  We  refer  to  the  values  {{/!„;,„}" =|  }A,=|  as  a 
sequence  of  inner  eigensteps  since  they  arise  from  matrices  of  inner  products  of  the  fj s 
(Gram  matrices),  whereas  outer  eigensteps  {{An;m}^=l}^=0  arise  from  sums  of  outer  products 
of  the  ffs  (frame  operators).  To  be  precise: 
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Definition  12.  Let  {A „}^=1  and  {fi„}„=1  be  nonnegative  nonincreasing  sequences.  A 
corresponding  sequence  of  inner  eigensteps  is  a  sequence  of  sequences  {{d„;m}”  =1}^=1  which 
satisfies  the  following  three  properties: 

(i)  The  final  sequence  is  {Tm}m=i:  AN-m  =  Am  for  every  m  =  1 ,N, 

(ii)  The  sequences  interlace:  {An-1;m}%~=\  E  {An;m}nm=l  for  every  n  =  2, . . .  ,N, 

(iii)  The  trace  condition  is  satisfied:  =1  Tn;m  =  X",=  i  for  every  n  =  1, . . . ,  N. 

Unlike  the  outer  eigensteps  of  Definition  1,  the  interlacing  relation  (ii)  here  involves 
two  sequences  of  different  length;  we  write  E  {/T«}(",=1  if  /3m+i  <  am  <  [3m  for  all 

m  =  1, . . . ,  n  -  1.  Now  we  revisit  Example  8  to  demonstrate  the  connection  between  inner 
and  outer  eigensteps: 

Example  13.  We  revisit  Example  8.  Here,  we  pad  {Tm}3  =1  with  two  zeros  so  as  to  match  the 
length  of  {/4}3=1.  That  is,  =  A2  =  d3  =  |,  A4  =  A5  =  0,  and/q  =  p2  =  p3  =  p4  =  p5  =  1. 
We  find  every  sequence  of  inner  eigensteps  {{T„;m}"=1}^=1,  namely  every  table  of  the  form: 

n  1  2  3  4  5 
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f  <  A5.3  <  A4:2  <  A3;  1  <  A4;  1  <  d5;i  =  |  and  so  A4;2  =  A3;1  =  A4;1  =  |,  yielding: 

77  1  2  3  4  5 


5 

d»;4 

4;;;3 

4/;;2 

4„;1  ? 


? 

? 


? 

? 

5 

3 


0 

? 

5 

3 

5 

3 


0 


0 


5 

3 

5 

3 

5 

3 


(4.2) 


Meanwhile,  since  /u„,  =  1  for  all  m,  the  trace  conditions  (iii)  give  that  the  values  in  the  /7th 
column  of  (4.2)  sum  to  n.  Thus,  Ti;i  =  1  and  A4-3  = 


and  A2;  i : 


77 

l 

2 

3 

4 

5 

^ n;5 

0 

^n;  4 

0 

0 

^n;3 

? 

2 

3 

5 

3 

^n;2 

? 

? 

5 

3 

5 

3 

^n;l 

l 

? 

5 

3 

5 

3 

5 

3 

label  A3;3  as 

*  and  A2:2 

as  y. 

at  which  point  (iii)  uniquely  determines  T3;2 

n 

1 

2 

3 

4 

5 

^n;  5 

0 

4 

0 

0 

(4.3) 

^ n;3 

X 

2 

3 

5 

3 

^ n;2 

y 

4  _  r 

3  X 

5 

3 

5 

3 

^n',1 

1 

2-y 

5 

3 

5 

3 

5 

3 
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For  our  particular  choice  of  {An}5n=l  and  {pn}5n=v  the  above  argument  shows  that  every 
corresponding  sequence  of  inner  eigensteps  is  of  the  form  (4.3).  Conversely,  one  may 
immediately  verify  that  any  {{An,m}"n=l}5n=l  of  this  form  satisfies  (i)  and  (iii)  of  Definition  12 
and  moreover  satisfies  (ii)  when  n  =  5.  However,  in  order  to  satisfy  (ii)  for  n  =  2,3,4,  x 
and  y  must  be  chosen  so  that  they  satisfy  the  ten  inequalities: 


{Avjn}m=l  E  {A4;m}m=l 

0<.v<|<|-.v<f, 

{A2;m}m=l  — 

x<y<\-x<2-y<\. 

(4.4) 

{4]  ;m(m=|  E  {d.2;m}m=l 

y  <  1  <  2  -  y. 

A  quick  inspection  reveals  the  system  (4.4)  to  be  equivalent  to  the  one  derived  in  the  outer 
eigenstep  formulation  (3.27)  presented  in  Example  8,  which  is  reducible  to  0  <  x  <  |, 
max{|,.r}  <  y  <  min{|  +  x,  |  -  x\.  Moreover,  we  see  that  the  outer  eigensteps  (3.26) 
that  arise  from  {di,  43}  =  {|,|,  |}  and  the  inner  eigensteps  (4.3)  that  arise  from 
{di,d2,d3,d4,  d5}  =  { f ,  f ,  §,0,0}  are  but  zero-padded  versions  of  each  other;  the  next  result 
claims  that  such  a  result  holds  in  general. 

Theorem  14.  Let  {d„}^=1  and  {/r„}^=1  be  nonnegative  and  nonincreasing,  and  choose  any 
M  <  N  such  that  An  =  0  for  every  n  >  M.  Then,  every  choice  of  outer  eigensteps 
(Definition  1)  corresponds  to  a  unique  choice  of  inner  eigensteps  (Definition  12)  and  vice 
versa,  the  two  being  zero-padded  versions  of  each  other. 

Specifically,  a  sequence  of  outer  eigensteps  {{d„;m)“=I  }^=()  gives  rise  to  a  sequence  of 
inner  eigensteps  {{An-m}"n=l}^=l,  where  An  m  :=  0  whenever  m  >  M.  Conversely,  a  sequence 
of  inner  eigensteps  {{An-m}'(n=l}^=l  gives  rise  to  a  sequence  of  outer  eigensteps  {{An-m}^=l}^=0, 
where  An-m  0  whenever  m  >  n. 

Moreover,  {An;m}^=1  is  the  spectrum  of  the  frame  operator  Fn  F*  ofFn  =  {/m}”=1  if  and 
only  if  {An-m}'lln=l  is  the  spectrum  of  the  Gram  matrix  F*Fn. 
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Proof.  First,  take  outer  eigensteps  {R,;m}^=1}^=0,  and  consider  {{T„;m}"=1}^=1,  where  we 
define 

An;m  =  0  whenever  m  >  M.  (4.5) 

Then  Definition  12.(i)  follows  from  Definition  l.(ii)  in  the  case  when  m  <  M.  and  follows 
from  (4.5)  and  the  assumption  that  A„  =  0  for  every  n  >  M  in  the  case  when  m  >  M.  Next, 
we  note  that  Definition  1  .(iii)  gives 


/^7i;m+ 1  —  ^n-X^m  —  ^-n^m  Vflt  —  1,  . 

..,M—  1, 

(4.6) 

0  <  An-i;M  <  Au-m, 

(4.7) 

for  every  n  =  1, . . . ,  N.  To  prove  Definition  12.(ii),  pick  any  n  =  2, ...  ,N.  We  need  to  show 
An:m+ 1  <  An-i;m  <  An;m  for  every  m  -  1, ...  ,n  -  1.  This  follows  directly  from  (4.6)  when 
n  <  M  or  when  n  >  M  and  m  <  M.  If  n  >  M  and  m  =  M.  then  (4.5)  and  (4.7)  together  give 


d^M+l  —  0  <  An-1-M  <  d„;M- 

Also,  (4.5)  gives  that  An;m+1  <  T„_  1;m  <  An;m  becomes  0  <  0  <  0  whenever  n  >  M  and 
m  >  M.  Next,  to  show  that  {{T„;m}"=1}w=1  satisfies  Definition  12. (iii),  note  that  when  n  >  M, 
(4.5)  and  Definition  l.(iv)  together  give 


M 


. =  =  z 


m=  1 


m- 1 


m- 1 


(4.8) 


Furthermore,  if  n  <  M ,  then  Definition  l.(i)  and  Definition  l.(iii)  together  give 


An-m  =  0  whenever  m  >  n. 


(4.9) 


and  so  (4.9)  and  Definition  l.(iv)  together  give  (4.8). 

Now  take  inner  eigensteps  {{/l„;m}”  =1}^=1,  and  consider  {{d„;m}"=1}^=0,  where  we  define 

A, );m  =  0  whenever  in  >  n.  (4.10) 
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Then  Definition  l.(i)  follows  directly  from  (4.10)  by  taking  n  =  0.  Also,  Definition  l.(ii) 
follows  from  Definition  12. (i)  since  M  <  N.  Next,  Definition  12. (ii)  gives 

An;m+l  —  4 n—l;m  —  4 n\m  All  2,  .  .  .  ,  N ,111  1,  .  .  .  ,  It  1.  (4.1 1) 

Using  the  nonnegativity  of  {An}^=l  along  with  Definition  12. (i)  and  an  iterative  application 
of  the  left-hand  inequality  of  (4.11)  then  gives 


0  <  An  -  An;n  <  ■  ■  ■  <  An-n  Vn  -  1, . . . ,  N. 


Combining  this  with  an  iterative  application  of  the  right-hand  inequality  of  (4.11)  then 
gives 

0  <  Am;m  <  ■  ■  ■  <  An;m  in  >  m .  (4.12) 

For  Definition  l.(iii),  we  need  to  show  (4.6)  and  (4.7)  for  every  n  =  1, . . . ,  N .  Considering 
(4.10),  when  n  -  1,  (4.6)  and  (4.7)  together  become  Ti;i  >  0,  which  follows  from  (4.12). 
Also,  when  n  >  M,  (4.11)  immediately  gives  (4.6),  while  (4.7)  follows  from  both  (4.12) 
and  (4.11): 

0  <  An-M+ 1  <  An-\:M  <  An-M- 


For  the  case  2  <  n  <  M,  note  that  (4.1 1)  gives  the  inequalities  in  (4.6)  whenever  m  <  n  —  1. 
Furthermore  when  m  =  n,  (4.10)  gives  An;n+ 1  =  An- \:n  =  0,  and  so  the  inequalities  in  (4.6) 
become  An;n  >  0,  which  follows  from  (4.12).  Otherwise  when  m  >  n,  the  inequalities  in 

(4.6)  become  0  <  0  <  0  by  (4.10).  To  finish  the  case  2  <  n  <  M,  we  need  to  prove  (4.7). 
When  n  =  M,  (4.7)  becomes  An;n  >  0,  which  follows  from  (4.12).  Otherwise  when  n  <  M , 

(4.7)  becomes  0  <  0  <  0  by  (4.10). 

At  this  point,  all  that  remains  to  be  shown  is  that  {{An-m}ffi=]  (^=()  satisfies  Definition  l.(iv). 
For  n  <  M,  (4.10)  and  Definition  12.(iii)  together  imply 

M  n  Yl 


^  ^  ^ n;m  ^  ^  ^  ^ 


m- 1 


m- 1 


(4.13) 
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Next,  note  that  (4.12),  Definition  12.(i),  and  our  assumption  that  An  =  0  for  every  n  >  M 
gives 

0  <  An;m  <  AN;m  =  Am  =  0  whenever  n  >  m  >  M. 

Thus,  An;m  =  0  whenever  n  >  m  >  M\  when  n  >  M,  we  can  combine  this  with 
Definition  12.(iii)  to  get  (4.13).  □ 

This  result,  when  coupled  with  a  complete  constructive  characterization  of  all  valid 
inner  eigensteps  provides  a  systematic  method  for  constructing  any  and  all  valid  outer 
eigensteps,  thereby  making  Step  A  of  Theorem  2  explicit. 

4.2  Top  Kill  and  the  existence  of  eigensteps 

In  Chapter  3,  Theorem  7  gave  us  an  explicit  construction  of  all  sequences  of  vectors 
whose  partial-frame-operator  spectra  match  the  eigensteps  chosen  in  Step  A  of  Theorem  2. 
We  now  focus  on  the  problem  of  performing  Step  A  explicitly,  namely  choosing  a  sequence 
of  outer  eigensteps  {{T„;m}^=1}^=0  which  satisfy  Definition  1.  By  Theorem  14,  we  see  that 
every  sequence  of  outer  eigensteps  corresponds  to  a  unique  sequence  of  inner  eigensteps. 
Note  that  if  a  sequence  of  inner  eigensteps  exists,  then  {d„}^=1  necessarily  majorizes  {pn}„=l 
since 

n  n  n 

^  .  A in  —  ^  ^  d«;m  —  ^  >n  ■ 
m=  1  m=  1  m=  1 

In  this  section,  we  prove  the  converse,  specifically  that  if  {An}^=l  majorizes  {ju«}^=1,  then  a 
corresponding  sequence  of  inner  eigensteps  {{T„;m}”=1}^=1  exists.  To  be  clear,  the  Schur- 
Horn  Theorem  gives  that  this  set  is  nonempty  if  and  only  if  {An}N=1  majorizes  {/un}^=v 
Horn  and  Johnson  [33]  already  proved  that  such  a  sequence  exists,  however,  their  proof 
provides  no  intuition  as  to  how  to  explicitly  construct  such  an  interlacing  sequence.  The 
main  algorithm  of  this  section,  Top  Kill ,  not  only  provides  an  alternative  proof  of  Horn  and 
Johnson’s  approach,  but  it  refines  Step  A  of  Theorem  2  by  giving  an  explicit  construction  of 
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a  feasible  set  of  eigensteps.  In  the  next  section,  we  use  these  results  to  further  parametrize 
the  set  of  all  eigensteps.  We  now  motivate  the  Top  Kill  algorithm  with  an  example: 


Example  15.  Let  N  =  3,  {A\,  A2,  /I3}  =  and  {ni,/u2,n2}  =  {1,1,1}.  Since  this 

spectrum  majorizes  these  lengths,  we  claim  that  there  exists  a  corresponding  sequence  of 
inner  eigensteps  {{T„;m}”=1}^=1.  That  is,  recalling  Definition  12,  we  claim  it  is  possible 
to  find  values  {/li;i}  and  {A2;i,A2;2}  which  satisfy  the  interlacing  requirements  (ii)  that 
{Ti;i}  E  {A2-a,A2-2}  E  {\->\->\)  as  well  as  the  trace  requirements  (iii)  that  Ai;i  =  1  and 
A2- ]  +  A2’2  =  2.  Indeed,  every  such  sequence  of  eigensteps  is  given  by  the  table: 


1  2  3 


An„  3 

An;  2 
An;  1 

where  x  is  required  to  satisfy 


x 

2  -  x 


1 

2 

3 

4 

7 

4 


\<  x  <\<2-  x  <\,  x  <  1  <  2  -  x. 


(4.14) 


(4.15) 


Clearly,  any  x  6  [|,  |]  will  do.  However,  when  N  is  large,  the  table  analogous  to  (4.14)  will 
contain  many  more  variables,  leading  to  a  system  of  inequalities  which  is  much  larger  and 
more  complicated  than  (4.15).  In  such  settings,  it  is  not  obvious  how  to  construct  even  a 
single  valid  sequence  of  eigensteps.  As  such,  we  consider  this  same  simple  example  from  a 
different  perspective — one  that  leads  to  an  eigenstep  construction  algorithm  which  is  easily 
implementable  regardless  of  the  size  of  N. 

The  key  idea  is  to  view  the  task  of  constructing  eigensteps  as  iteratively  building  a 
staircase  in  which  the  nth  level  is  An  units  long.  For  this  example  in  particular,  our  goal  is  to 
build  a  three-step  staircase  where  the  bottom  level  has  length  |,  the  second  level  has  length 
|,  and  the  top  level  has  length  \\  the  profile  of  such  a  staircase  is  outlined  in  black  in  each 
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3  3  3 


AAA 
(d)  (e)  (f) 

Figure  4.1:  Two  attempts  at  iteratively  building  a  sequence  of  inner  eigensteps  for 
{Ai,A2,A3}  =  {|,  and  {1*1,1*2,11-3}  =  {1,1, 1}.  As  detailed  in  Example  15,  the  first  row 
represents  a  failed  attempt  in  which  we  greedily  complete  the  first  level  before  focusing  on 
those  above  it.  The  failure  arises  from  a  lack  of  foresight:  the  second  step  does  not  build 
sufficient  foundation  for  the  third.  The  second  row  represents  a  second  attempt,  one  that  is 
successful.  There,  we  begin  with  the  final  desired  staircase  and  work  backwards.  That  is, 
we  chip  away  at  the  three-level  staircase  (d)  to  produce  a  two-level  one  (e),  and  then  chip 
away  at  it  to  produce  a  one-level  one  (f).  In  each  step,  we  do  this  by  removing  as  much 
as  possible  from  the  top  level  before  turning  our  attention  to  the  lower  levels,  subject  to 
the  interlacing  constraints.  We  refer  to  this  algorithm  for  iteratively  producing  {An- 
from  {An.m}’;n=l  as  Top  Kill.  Theorem  16  shows  that  Top  Kill  will  always  produce  a  valid 
sequence  of  eigensteps  from  any  desired  spectrum  {T„}w=1  that  majorizes  a  given  desired 
sequence  of  lengths  {f*n}^=v 


of  the  six  subfigures  of  Figure  4. 1 .  The  benefit  of  visualizing  eigensteps  in  this  way  is  that 
the  interlacing  and  trace  conditions  become  intuitive  staircase-building  rules.  Specifically, 
up  until  the  /7th  step,  we  will  have  built  a  staircase  whose  levels  are  {An- i-,m}n~}v  To  build 
on  top  of  this  staircase,  we  use  n  blocks  of  height  1  whose  areas  sum  to  i*n.  Each  of  these 
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n  new  blocks  is  added  to  its  corresponding  level  of  the  current  staircase,  and  is  required  to 
rest  entirely  on  top  of  what  has  been  previously  built.  This  requirement  corresponds  to  the 
interlacing  condition  (ii)  of  Definition  12,  while  the  trace  condition  (iii)  corresponds  to  the 
fact  that  the  block  areas  sum  to  jjn. 

This  intuition  in  mind,  we  now  try  to  build  such  a  staircase  from  the  ground  up.  In  the 
first  step  (Figure  4.1(a)),  we  are  required  to  place  a  single  block  of  area  /q  =  1  on  the  first 
level.  The  length  of  this  first  level  is  Ai-i  =  Hi-  In  the  second  step,  we  build  up  and  out 
from  this  initial  block,  placing  two  new  blocks — one  on  the  first  level  and  another  on  the 
second — whose  total  area  is  /r2  =  1-  The  lengths  A2. 1  and  A2- 2  of  the  new  first  and  second 
levels  depends  on  how  these  two  blocks  are  chosen.  In  particular,  choosing  first  and  second 
level  blocks  of  area  |  and  |,  respectively,  results  in  {d2;i,  T2;2}  =  1 1,  |}  (Figure  4.1(b));  this 
corresponds  to  a  greedy  pursuit  of  the  final  desired  spectrum  {\,\,\},  fully  completing  the 
first  level  before  turning  our  attention  to  the  second.  The  problem  with  this  greedy  approach 
is  that  it  doesn’t  always  work,  as  this  example  illustrates.  Indeed,  in  the  third  and  final  step, 
we  build  up  and  out  from  the  staircase  of  Figure  4.1(b)  by  adding  three  new  blocks — one 
each  for  the  first,  second  and  third  levels — whose  total  area  is  /q  =  1.  However,  in  order 
to  maintain  interlacing,  the  new  top  block  must  rest  entirely  on  the  existing  second  level, 
meaning  that  its  length  /13;3  <  A2:2  =  \  cannot  equal  the  desired  value  of  That  is,  because 
of  our  poor  choice  in  the  second  step,  the  “best”  we  can  now  do  is  {T3;i,  /l3;2,/l3;3}  =  {|,  1, 
(Figure  4.1(c)): 

n  12  3 

An;3  4 

d„;2  \  1 

A«l  1  |  | 

The  reason  this  greedy  approach  fails  is  that  it  doesn’t  plan  ahead.  Indeed,  it  treats  the 
bottom  levels  of  the  staircase  as  the  priority  when,  in  fact,  the  opposite  is  true:  the  top 
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levels  are  the  priority  since  they  require  the  most  foresight.  In  particular,  for  /13;3  to  achieve 
its  desired  value  of  \  in  the  third  step,  one  must  lay  a  suitable  foundation  in  which  A2.2  >  k 
in  the  second  step. 

In  light  of  this  realization,  we  make  another  attempt  at  building  our  staircase.  This 
time  we  begin  with  the  final  desired  spectrum  {T3;i,  d3;2,  d3;3}  =  {\,  |,  (Figure  4.1(d)) 
and  work  backwards.  From  this  perspective,  our  task  now  is  to  remove  three  blocks — the 
entirety  of  the  top  level,  and  portions  of  the  first  and  second  levels — whose  total  area  is 
/r3  =  1.  Here,  the  interlacing  requirement  translates  to  only  being  permitted  to  remove 
portions  of  the  staircase  that  were  already  exposed  to  the  surface  at  the  end  of  the  previous 
step.  After  lopping  off  the  top  level,  which  has  area  d3;3  =  (,  we  need  to  decide  how  to 
chip  away  /i  1  -  T3;3  =  1  -  \  \  units  of  area  from  the  first  and  second  levels,  subject  to 

this  constraint.  At  this  point,  we  observe  that  in  the  step  that  follows,  our  first  task  will 
be  to  remove  the  remaining  portion  of  the  second  level.  As  such,  it  is  to  our  advantage  to 
remove  as  much  of  the  second  level  as  possible  in  the  current  step,  and  only  then  turn  our 
attention  to  the  lower  levels.  That  is,  we  follow  Thomas  Jefferson’s  adage,  “Never  put  off 
until  tomorrow  what  you  can  do  today.”  We  dub  this  approach  Top  Kill  since  it  “kills”  off 
as  much  as  possible  from  the  top  portions  of  the  staircase.  For  this  example  in  particular, 
interlacing  implies  that  we  can  at  most  remove  a  block  of  area  ^  from  the  second  level, 
leaving  |  units  of  area  to  be  removed  from  the  first;  the  resulting  two-level  staircase — the 
darker  shade  in  Figure  4.1(e) — has  levels  of  lengths  {/Ki*^}  =  {§,  |}.  In  the  second  step, 
we  then  apply  this  same  philosophy,  removing  the  entire  second  level  and  a  block  of  area 
/i 2  -  T2;2  =  \  -  \  =  \  from  the  first,  resulting  in  the  one-level  staircase  (Figure  4.1(f))  in 
which  {di;i}  =  1.  That  is,  by  working  backwards  we  have  produced  a  valid  sequence  of 
eigensteps: 
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n 


1 


2 


3 


^■n; 3  4 

3  1  3 

An;2  2  4 

1  1  I 

The  preceding  example  illustrated  a  systematic  “Top  Kill”  approach  for  building 

eigensteps;  we  now  express  these  ideas  more  rigorously.  As  can  be  seen  in  the  bottom 
row  of  Figure  4.1,  Top  Kill  generally  picks  T„_i;m  :=  An;m+l  for  the  larger  m’s.  Top  Kill 
also  picks  T„_  1;m  :=  An;m  for  the  smaller  m’s.  The  level  that  separates  the  larger  m’s  from 
the  smaller  m’s  is  the  lowest  level  from  which  a  nontrivial  area  is  removed.  For  this  level, 
say  level  k,  we  have  An-k+ i  <  pn  <  Airk.  In  the  levels  above  k,  we  have  already  removed 
a  total  of  An;k+ 1  units  of  area,  leaving  pn  -  An-k+ 1  to  be  chipped  away  from  An;k,  yielding 
/l„_i;/(  :=  /l„;/c  -  (yu„  -  An:k+  i).  The  following  theorem  confirms  that  Top  Kill  always  produces 
eigensteps  whenever  it  is  possible  to  do  so: 


Theorem  16.  Suppose  {An-m}"n=l  >  {pm\”l=],  and  define  {An-]:mY‘nJ]  according  to  Top  Kill, 
that  is,  pick  any  k  such  that  An;k+\  <  pn  <  An;k,  and  for  each  m  =  1 —  1,  define: 


An.m,  l  <  m  <  k  -  1, 

An\k  "3"  An_k+i  Pn ,  m  —  k, 

An;m+ 1,  k+l<m<n-l. 


(4.16) 


Then  {T„_ E  {An;m}nm=x  and  {/l„-i;m("J,  >  \Pm\nmly 

Furthermore,  given  any  nonnegative  nonincreasing  sequences  {An}^=x  and  {pn}„=i 
such  that  {/1„}^=|  >  {Pn)n=\>  define  AN-m  :=  Am  for  every  m  =  1  and  for  each 

n  =  N, . . .  ,2,  consecutively  define  {An- according  to  Top  Kill.  Then  {{d,i;„,}J';=1}^=1 
are  inner  eigensteps. 


Proof.  For  the  sake  of  notational  simplicity,  we  denote  {am}nm}x  :=  {An-v,m}nm=i  and 
{A»}"i=  1  :=  Wn;mZ=v  since  {/?»>}",= i  ^  {PmYm=  v  we  necessarily  have  that/?,,  <  pn  <  px  <  /?, 
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and  so  there  exists  k  =  1, . . .  ,n  -  1  such  that  /3k+\  <  fin  <  f3k.  Though  this  k  may  not 
be  unique  when  subsequent  {3m’s  are  equal,  a  quick  inspection  reveals  that  any  appropriate 
choice  of  k  will  yield  the  same  crm’s,  and  so  Top  Kill  is  well-defined.  To  prove  {am}^=\  E 
{fim}nm=v  we  need  to  show  that: 

fim+1  <am</3m  (4.17) 

for  every  m  =  -  1.  If  1  <  m  <  k  —  1,  then  am  :=  f3m,  and  so  the  right- 

hand  inequality  of  (4.17)  holds  with  equality,  at  which  point  the  left-hand  inequality  is 
immediate.  Similarly,  if  k  +  1  <  m  <  n  —  1,  then  am  :  =  and  so  (4.17)  holds  with 
equality  on  the  left-hand  side.  Lastly  if  m  =  k,  then  ak  :=  (3k  +  [3k  f ,  ~/jn,  and  our  assumption 
that  {3k+l  <  nn  <  f3k  gives  (4.17)  in  this  case: 

fik+ 1  —  [3k  “I"  [3  k  --  1  fin  —  /3k  • 

Thus,  Top  Kill  produces  {o'm}"Tl|  such  that  E  \J3mjnm=r  We  next  show  that 

^  Mlnlv  Tfj<k-1,  then  since  {[3,n}nm=l  >  {fim}nm=v  we  have: 

i  j  j 

^  ^  ^  ^  Pm  —  ^  ^  f^m  • 

m=  1  m=  1  m=  1 

On  the  other  hand,  if  j  >  k ,  we  have: 

7  l  j  7+1 

^  ^  ^  ^  Pm  (Pk  Pk+ 1  /^ti)  ^  ^  Pm+ 1  ^  ^  Pm  (4*  1 8) 

m=l  m=l  m=&+l  m=l 

with  the  understanding  that  a  sum  over  an  empty  set  of  indices  is  zero.  We  continue  (4.18) 
by  using  the  facts  that  {pm}nm=l  >  {l*m}nm=i  and  /ij+1  >  /jn: 

j  7+1  7+1  7 

^  ^  ^  ^  Pm  Hn  —  ^  ^  H-m  Hn  —  ^  ^  /^m •  (4. 19) 

771=1  772=1  771=1  771=1 

Note  that  when  j  =  n,  the  inequalities  in  (4.19)  become  equalities,  giving  the  final  trace 
condition. 

For  the  final  conclusion,  note  that  one  application  of  Top  Kill  transforms  a  sequence 
Un;m}m=t  +at  majorizes  {fim}nm={  into  a  shorter  sequence  {/l„_i;m)"'ll1  that  interlaces  with 
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{^n\m)nm=  |  and  majorizes  {/im \"~J] .  As  such,  one  may  indeed  start  with  AN-m  :=  Am  and 
apply  Top  Kill  N  -  1  times  to  produce  a  sequence  {{T„;m}”=1}^=1  that  immediately  satisfies 
Definition  12.  □ 

4.3  Parametrizing  eigensteps 

In  the  previous  section,  we  discussed  Top  Kill,  an  algorithm  designed  to  construct 
a  sequence  of  inner  eigensteps  from  given  nonnegative  nonincreasing  sequences  {An}^=l 
and  {/i„}^=1.  As  previously  mentioned,  the  Schur-Horn  Theorem  gives  that  the  set  of  all 
inner  eigensteps  is  nonempty  if  and  only  if  {An}^=l  majorizes  {/v(^=|.  Indeed,  as  noted 
at  the  beginning  of  the  previous  section,  if  such  a  sequence  of  eigensteps  exists  then  we 
necessarily  have  that  {/1„}^=1  >  \jj„}^=l.  Conversely,  if  {/!„};'  ,  >  {yu„}^=1  then  Theorem  16 
states  that  Top  Kill  will  produce  a  valid  sequence  of  eigensteps  from  {An}^=1  and  \/jH}^=r 
That  is,  the  results  of  the  previous  section  give  an  alternative  proof  of  the  Schur-Horn 
Theorem. 

In  this  section,  we  use  the  intuition  underlying  Top  Kill  to  find  a  systematic  method 
for  producing  all  such  eigensteps.  If  the  values  {{An;mY‘n=]  are  treated  as  independent 
variables,  it  can  easily  be  shown  that  the  set  of  all  inner  eigensteps  for  a  given  {d„}^=1  and 
{Hn}N=x  form  a  convex  polytope  in  W7(n~1)/2.  The  main  result  of  this  section  gives  a  complete 
characterization  of  the  set  of  all  eigensteps  for  a  given  {An}^=l  and  {/./„()  Unlike  Top  Kill 
which  gives  just  one  strategy  for  constructing  a  sequence  of  eigensteps  required  by  Step  A 
of  Theorem  2,  the  results  of  this  section  provide  a  systematic  method  for  producing  any 
feasible  sequence  of  eigensteps.  In  the  work  that  follows,  we  view  these  non-Top-Kill- 
produced  eigensteps  as  the  result  of  applying  generalizations  of  Top  Kill  to  {An}^=l  and 

{rf=r 

Recall  Example  13  at  the  beginning  of  this  Chapter  where  {Ai,  A2,  A3,  A4,  /I5}  = 
{|,  |,  |,0,0}  and  /un  =  1  for  all  n  =  1, ...  ,5.  In  order  for  (4.1)  to  be  a  valid  sequence  of 
eigensteps,  the  10  unknown  values  in  (4.1)  must  satisfy  the  interlacing  and  trace  conditions 
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(ii)  and  (iii)  of  Definition  12.  The  set  of  all  such  eigensteps  form  a  convex  polytope 
in  M10.  Taking  advantage  of  the  interlacing  and  trace  conditions,  we  reduce  these  10 
unknowns  to  just  two  unknowns  in  (4.3).  The  variables  x  and  y  must  then  be  chosen  so 
that  they  satisfy  the  system  of  inequalities  given  in  (4.4).  In  general,  for  any  {Am}™=l  and 
iHn)^=  i,  constructing  all  sequences  of  eigensteps  will  require  simplifying  a  large  system  of 
inequalities  which  motivates  us  to  find  a  more  systematic  way  of  parametrizing  the  resulting 
convex  polytope. 

In  this  section,  we  suggest  a  different  method  for  parametrizing  the  polytope:  to 
systematically  pick  the  values  {{T„;m}"=1}^=1  one  at  a  time.  Top  Kill  is  one  way  to  do  this: 
working  from  the  top  levels  down,  we  chip  away  7/5  =  1  units  of  area  from  {A5-m}5m=i  to 
successively  produce  d4; 4  =  0,  d4;3  =  |,  A4;2  =  §  and  /14;1  =  we  then  repeat  this  process 
to  transform  {/t4;m}fn=1  into  {T3;m}^=1,  and  so  on;  the  specific  values  can  be  obtained  by 
letting  ( x,y )  =  (0,  |)  in  (4.3).  We  seek  to  generalize  Top  Kill  to  find  all  ways  of  picking  the 
An-jn  s  one  at  a  time.  As  in  Top  Kill,  we  work  backwards:  we  first  find  all  possibilities  for 
d4;4,  then  the  possibilities  for  d4;3  in  terms  of  our  choice  of  T4;4,  then  the  possibilities  for 
/14;2  in  terms  of  our  choices  of  d4;4  and  /14;3,  and  so  on.  That  is,  we  iteratively  parametrize 
our  convex  polytope  in  the  following  order: 


T4;4,  d4;3,  d4;2,  d4;l>  d3;3,  T3;2j  43;i,  A2-2,  A  2;1,  Tl;l. 

More  generally,  for  any  {/1„}^=1  and  {jun}^=l  such  that  {An}^=l  >  we  construct 

every  possible  sequence  of  eigensteps  {{An-m}nm =1}^=1  by  finding  all  possibilities  for  any  given 
An-i-jc  in  terms  of  A„';m  where  either  n  -  1  <  n'  or  n  -  1  =  n'  and  k  <  m.  Certainly,  any 
permissible  choice  for  An-i$  must  satisfy  the  interlacing  criteria  (ii)  of  Definition  12,  and  so 
we  have  bounds  An-k+\  <  <  Arrk.  Other  necessary  bounds  arise  from  the  majorization 

conditions.  Indeed,  in  order  to  have  both  {An;m}nm=1  >  {//„,}"  =1  and  {An-1;m}^=\  >  we 

need: 


n- 1 


77-1 


777=1  777=1  777=1  771=1 


(4.20) 
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and  so  we  may  view  /un  as  the  total  change  between  the  eigenstep  spectra.  Having 


already  selected  d„_ 1^+1,  we’ve  already  imposed  a  certain  amount  of  change 
between  the  spectra,  and  so  we  are  limited  in  how  much  we  can  change  the  kth  eigenvalue. 
Continuing  (4.20),  this  fact  can  be  expressed  as: 


n—  1 


n—  1 


-  v,„)  >  -t,„, + 


(4.21) 


m=l 


m—k 


where  the  inequality  follows  from  the  fact  that  the  summands  Amn  -  A„-  i;m  are  nonnegative 


if  {/l„_i  is  to  be  chosen  so  that  {d 


/I- 1  J 


l/i-l 


i  E  {d„;m}”=1.  Rearranging  (4.21)  then  gives 


a  second  lower  bound  on  to  go  along  with  our  previously  mentioned  requirement  that 

An:k+l  —  /- 1  :k  • 

/l  /l-l 

^  ^  Xn\m  ^  ^  l;m  —  X n—l;k  • 


m=k  m=k+l 

That  is,  we  necessarily  have  that 


71 —  1 


max  |  1  ?  ^  ^  Xn-m  ^  ^  ^/i— l;m  /^/i  |  —  ^/i-l;^* 


(4.22) 

m=k  m=k+ 1 

We  next  apply  the  intuition  behind  Top  Kill  to  obtain  other  upper  bounds  on  Xn-\ to 
go  along  with  our  previously  mentioned  requirement  that  \.k  <  An;Jc.  Recall  that  we  have 
already  selected  {An-mYll~Jk+i  and  are  attempting  to  find  all  possible  choices  An-i-k  that  will 
allow  the  remaining  values  to  be  chosen  in  such  a  way  that: 


Un-l;m}nml\  E  UWJUl,  Un-l;,n}Z\  > 


(4.23) 


To  do  this,  we  recall  our  staircase-building  intuition  from  the  previous  section:  if  it  is 
possible  to  build  a  given  staircase,  then  one  way  to  do  this  is  to  assign  maximal  priority 
to  the  highest  levels,  as  these  are  the  most  difficult  to  build.  As  such,  for  a  given  choice 
of  An-i-k,  if  it  is  possible  to  choose  {d„_ in  such  a  way  that  (4.23)  holds,  then  it  is 
reasonable  to  expect  that  one  way  of  doing  this  is  to  pick  d„_  by  chipping  away  as 
much  as  possible  from  An-k- 1,  then  pick  d„_i;Jt_2  by  chipping  away  as  much  as  possible  from 
An;k_ 2,  and  so  on.  That  is,  we  pick  some  arbitrary  value  d„_ \-k  and  to  test  its  legitimacy, 
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apply  the  Top  Kill  algorithm  to  construct  the  remaining  undetermined  values  {d =\; 
we  then  check  whether  or  not  E  {An-m}nm=v 


To  be  precise,  note  that  prior  to  applying  Top  Kill,  the  remaining  spectrum  is  {An;m}km  =\ 
and  that  the  total  amount  we  will  chip  away  from  this  spectrum  is: 


ri/i  ( 'f; :n  "t"  ^  ^ ^ njn  An-\ln)j.  (4.24) 

m-k 

To  ensure  that  our  choice  of  satisfies  A,rk  <  d„_i;*_i,  we  artificially  reintroduce  Airk 

to  both  (4.24)  and  the  remaining  spectrum  {An;m}km i'j  before  applying  Top  Kill.  That  is,  we 
apply  Top  Kill  to  {J3m}nm=l  :=  {d„;m}*=1  U  {0}nm=k+l  where: 

n— 1  n  n- X 

H  *  —  “I”  ^  4^;^  —  Hn  ^  ^  ^  ^  ^n-X^m-  (4*25) 

m=k  m=k+ 1  m—k 

Specifically  in  light  of  Theorem  16,  in  order  to  optimally  subtract  /i  units  of  area  from 
{Pm}nm=v  we  first  pick  j  such  that  /3j+\  <  <  (3j.  We  then  use  (4.16)  to  produce  a  zero- 

padded  version  of  the  remaining  new  spectrum  {An^.m\k~^  U  {0}''/=/(: 


ft;fti? 

1  <  m  <  j  -  1, 

ft 

ft-1 

•Tl\j  4ft  ;j+  J 

/^ft  ^  ^  4^-m' 

m  =  j 

fti'=A:+l 

m'=k 

-ft;/ft+l  ? 

j  +  1  <  m  <  k  -  1 

Picking  /  such  that  j  +  1  <  /  <  k,  we  now  sum  the  above  values  of  d„_ i;m  to  obtain: 

;-i  j-i  /-i 

)  'i/;- 1  :m  —  A, ;-l;m  +  d7?_  I :/  +  ^  ^  '1  >i—  1  :m 


m=l  ra=&+l  m=k 

n  n 

Adding  X  l^m  X  /l„;m  =  0  to  the  right-hand  side  of  (4.26)  then  yields: 


(4.26) 


m— 1  m= 1 

/-I 


l— l  l  n  n- 1  «  w 

^  ^  ^  ^  4^^  jJ,n  +  ^  ^  ^  ^  ^n-X^m  ^  ^  Urn  ^  ^  4^ 

7ft=l  fti=l  m=k+ 1  /ft=&  fti=l  m—X 


(4.27) 


m=l  tfi—lA- 1 
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Now,  in  order  for  {An^m}nmf:x  >  {/r,„)"  J,  as  desired,  (4.27)  must  satisfy: 


/-i 


/-i 


n- 1 


n—  1 


X>„<  2  Z  'i— -Z'1—-- 


(4.28) 


m=l  m=  1  m=l  m=/+l 

Solving  for  in  (4.28)  then  gives: 

n—  1  A:  ft— 1 


m—k 


v,*  <  ^  - x  v,„. 


(4.29) 


m=l  m=l+ 1 


m=^+l 


Note  that,  according  to  how  we  derived  it,  (4.29)  is  valid  when  j  +  1  <  /  <  k.  Combining 
the  interlacing  requirement  that  An-i;k  <  with  (4.29),  we  have 

n-1  A-  «-l 

^  min|/ln;^,  min  ^  ^  ",  ^  ^  dn;m  ^  ^  (4.30) 

m=/  m=/+ 1  m=^+l 


As  established  in  the  following  theorem,  this  bound  actually  holds  when  /  =  1 
Overall,  (4.22)  and  (4.30)  are  precisely  the  bounds  that  we  verify  in  the  following  result: 


Theorem  17.  Suppose  {An-m}nm=x  >  {pm\'n=v  Then  {An-1;m}^=\  >  {//,„}”  }x  and  {An-1;m}^=\  E 
{An;m}nm=1  if  and  only  if  An-  1;k  €  Bn^;k]for  every  k  =  1, 1,  w/zcrc; 


-l;m  l^n  f  ? 


m=k  m=k+ 1 

ft-1  &  /t- 1 

Bn—\\k  • —  nun|/lft;^^min^  |  ^  ^  Pm  ^  ^  ^  ^  ^w— i;j 


m=/  fti=/+l 


fti=A:+l 


(4.31) 

(4.32) 


//ere,  we  ase  £/ze  convention  that  sums  over  empty  sets  of  indices  are  zero.  Moreover, 
suppose  /lft-i;ft-i, . . .  ,/lft-i;£+i  are  consecutively  chosen  to  satisfy  these  bounds.  Then 
An-v,k  <  and  so  Xn-\^  can  also  be  chosen  from  such  an  interval. 


Proof  For  the  sake  of  notational  simplicity,  we  let  {am }”=\  :=  {A„-1;m}^=\,  {/?,„}"I=1  :  = 
{ An-m}m=x,  A k  ■—  and  Bk  Bn_ \  k. 

(=>)  Suppose  {«,„(":*  >  IPmYfJt  and  {am}n~}x  E  { /3m}nm=v  Fix  any  particular  k  = 
1,  ...,n- 1.  Note  that  interlacing  gives  /C+,  <  ak  <  ff;,  which  accounts  for  the  first  entries  in 
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(4.31)  and  (4.32).  We  first  show  ak  >  Ak.  Since  {/?m}"=1  >  {nm)nm=l  and  {orin}"=\  >  {^"J, , 


then: 

n  n—  1  77  77- 1  77- 1 

l^n  ~  ^  ^  Hm  ^  ^  Hm  —  ^  ^  Pm  ^  ^  Pn  ^  (Pm  &m)-  (4.33) 

m=l  777=1  m=l  m=l  m=l 

Since  {crm}"^  E  ,  the  summands  in  (4.33)  are  nonnegative,  and  so: 

n— 1  n  n— 1 

Hn  —  Pn  ^  ^(Pm  ^m)  ^  ^ Pm  ^  ^  &m  (4.34) 

m—k  m=k  m=k+ 1 

Isolating  ak  in  (4.34)  and  combining  with  the  fact  that  ak  >  Pk+\  gives  ak  >  Ak.  We  next 
show  that  ak  <  Bk.  Fix  /  =  1  Then  {crm}”“\  >  {nJ^  implies  Y!mi\  1  Bm 

and  Xm='i  am  -  Zr='i  Bmi  and  so  subtracting  gives: 

77-  1  77-1  77-1  &-1  77-1  &— 1 

^  ^  l^lll  —  ^  ^  ^777  —  ^  ^  ^777  "I"  ^  ^  ^777  —  ^  ^  ^777  ^  j/^777+1  ?  (4.35) 

777=/  777=/  777=/:  777=/  lll—k  m=l 

where  the  second  inequality  follows  from  E  {/?,„}"  =1.  Since  our  choice  for 

/  =  1, . . . ,  k  was  arbitrary,  isolating  ak  in  (4.35)  and  combining  with  the  fact  that  ak  <  [3k 
gives  ak  <  Bk. 

(<=)  Now  suppose  Ak  <  ak  <  Bk  for  every  k  -  1, . . . ,  n  -  1.  Then  the  first  entries  in 
(4.31)  and  (4.32)  give^+i  <  ak  <  Pk  for  every  k  =  1 —  1 ,  that  is,  {am}^=\  E  {pm)nm=v 
It  remains  to  be  shown  that  .  Since  ak  <  Bk  for  every  k  -  1 -  1 , 

then: 

77-1  k  77-1 

ak  <  ^  Hm  -  ^  Pm  -  ^  ffm  Vfc  =  1, . . . ,  n  -  1,  l=l,...,k.  (4.36) 

777=/  777=/+ 1  m=k+ 1 

Rearranging  (4.36)  in  the  case  where  l  =  k  gives: 

77-1  77-1 

^  crm  <  ^  n,n  Vk  =  1 —  1 .  (4.37) 

m=k  m—k 

Moreover,  ai  >  A ,  implies  O',  >  YJh=iPm  ~  Y?m=2am  ~ Bn-  Rearranging  this  inequality  and 
applying  {/?,„}”  =1  >  {jum}"=1  then  gives: 

77-1  77  77-1 

^  ^  0^777  ^  ^  ^  Pm  /^7?  ^  ^  A^777  *  (4.38) 

777=1  777=1  777=1 
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Combining  (4.38)  with  (4.37)  in  the  case  where  k  =  1  gives: 

n—  1  72-1 

^  ^  ^  t^m •  (4.39) 

722=1  722=1 

Subtracting  (4.37)  from  (4.39)  completes  the  proof  that  {crm}”“11  > 

For  the  final  claim,  we  first  show  that  the  claim  holds  for  k  =  n  —  1,  namely  that 
A„_ i  <  Bn-\.  Explicitly,  we  need  to  show: 

(  ( "_1  "_1  1 1 

max{/?„,/?„_i  +  pn  -jjn\  <  min|/3„_i,;_min_  ,  Z  #»}}.  (4.40) 

722=/  722=/+ 1 

Note  that  (4.40)  is  equivalent  to  the  following  inequalities  holding  simultaneously: 

(i)  (3n  <  Pn-  U 

(ii)  Pn—  1  Pn  l^n  —  Pn-\i 

72-  1  72-1 

(iii)  Pn  —  ^  ^  l^m  ^  ^  Pm  V/—  1 ,  .  .  •  ,  ft  1, 

m—l  722=/+ 1 

72-1  72-1 

(iv )  Pn- 1  "1“  Pn  l^n  —  ^  ^  l^m  ^  ^  Pm  V/—  1 ,  .  .  .  ,  ft  1. 

722=/  722=/+ 1 

First,  (i)  follows  immediately  from  the  fact  that  {/3m}"  =1  is  nonincreasing.  Next,  rearranging 
(ii)  gives  pn  <  /un,  which  follows  from  {pm}nm=x  >  =1*  For  (iii),  the  facts  that 

{/U'Ui  >  {/4«C=1  and  {n,n}nm=l  is  nonincreasing  imply: 

72  72  72-1 

^  ^  Pm  —  ^  ^  /^772  —  ^  ^  /^722  ^  —  1  >  •  •  •  >  ft  1  ? 

722=/+ 1  722=/+ 1  722=/ 

which  in  turn  implies  (iii).  Also  for  (iv),  the  facts  that  {pm}nm=\  is  nonincreasing  and 

{#«}”=  1  >  ialP'y: 

72  72  72 

Pn- 1  "t"  ^  ^  Pm  —  A  j  j  /^m  ^  1 ,  .  .  .  ,  W  1 , 

m=/+l  m=/  m—l 

which  in  turn  implies  (iv).  We  now  proceed  by  induction.  Assume  ak+\  satisfies  Ak+\  < 
(Y/c+ 1  <  Bk+].  Given  this  assumption,  we  need  to  show  that  Ak  <  Bk.  Considering  the 
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definitions  (4.31)  and  (4.32)  of  Ak  and  Bk,  this  is  equivalent  to  the  following  inequalities 
holding  simultaneously: 

(i)  A+i  -  Pk, 

n  n—  1 

(ii)  ^  ^  Pm  ^  ^  Hn  —  Pki 

m—k  m=k+ 1 

77-1  A:  71-1 

(iii)  Pk+ 1  —  ^  ^  l^m  ^  ]  Pm  ^  ^  —  1* • •  •  »  &* 

iYi— l  m=l+X  m=k+ 1 

n  Ti-l  Ti-l  A:  Ti-l 

(iv)  ^  ^  Pm  ^  ^  ^Tii  l^n  —  ^  ^  f^m  ^  ^  Pm  ^  ^  ^tii  V/  —  1,  .  .  .  ,  k, 

m—k  m=k+\  m=l  m=l+ 1  ra=A:+l 

Again,  the  fact  that  {pm}nm =1  is  nonincreasing  implies  (i).  Next,  o^+i  >  A^+1  gives: 

71  71-1 

*^ &+ 1  —  ^  ^  Pm  ^  ]  &m  t^n  ? 

m=k+ 1  m=k+2 

which  is  a  rearrangement  of  (ii).  Similarly,  ak+x  <  Bk+\  gives: 

72-1  k+1  71-1 

®k+l  —  ^  ^  l^m  ^  ^  Pm  ^  j  ^Tii  ^  1 ,  .  .  .  ,  k  +  1 , 

772—/  771=/+ 1  m=k+2 

which  is  a  rearrangement  of  (iii).  Note  that  we  don’t  use  the  fact  that  (iii)  holds  when 
l  =  k  +  1.  Finally,  (iv)  follows  from  the  facts  that  {pm}nm=  {  is  nonincreasing  and  {pm}nm=\  > 
{ium}nm=v  since  they  imply: 

71  71  71 

Pk  +  ^  Pm  <  ^jPm  ^  ^  Bin  V/  =  1,  .  .  .  ,  k, 

771=/+ 1  771=/  771=/ 

which  is  a  rearrangement  of  (iv).  □ 

By  starting  with  a  sequence  {AN.m\Nm=]  =  {4m}"=|  that  majorizes  given  {//,„}^=1, 
repeatedly  applying  Theorem  17  to  construct  {/lw_i;m("7l1  from  {An;m}^=1  results  in  a 
sequence  of  inner  eigensteps.  Conversely,  any  sequence  of  inner  eigensteps  {{+„;„}" =1}^=1 
can  be  constructed  by  repeatedly  applying  Theorem  17.  This  fact  is  summarized  in  the 
following  corollary: 
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Corollary  18.  Let  {Xn)Nn={  and  {pn}^=i  be  nonnegative  and  nonincreasing  where  \An\^=]  > 
{fin}^=v  Every  sequence  of  inner  eigensteps  {{T„;m}”=1}^=1  can  be  constructed  by  the 
following  algorithm:  Let  An-m  =  Am  for  all  m  =  1, . . .  ,N- 1;  for  any  n  =  N, . . .  ,2  construct 
U«-I;m}”=l|  from  Un;mrm=1  by  picking  An_x.k  e  [An_ Uk,  Bn_];k\  for  all  k  =  n  -  1, . . . ,  1,  where 
A„_  1;*  and  i;k  are  given  by  (4.31)  and  (4.32),  respectively.  Moreover,  any  sequence 
constructed  by  this  algorithm  is  indeed  a  corresponding  sequence  of  inner  eigensteps. 

We  now  redo  Example  13  to  illustrate  that  Corollary  18  indeed  gives  a  more  systematic 
way  of  parametrizing  eigensteps: 


Example  19.  We  wish  to  parametrize  the  eigensteps  corresponding  to  UNTFs  of  5  vectors 
in  M3.  In  the  end,  we  will  get  the  same  parameterization  of  eigensteps  as  in  Example  13: 


n  1  2  3  4  5 

d„;  5  0 

An-4  0  0 

(4.41) 


}  v  ±  2. 

3  x  3  3 

r  „  4  _  v  5  5 

An-,2  y  3  X  3  3 

An;  1  1  2  -  V  3  3  3 

where  0  <  x  <  max{|,^:}  <  y  <  min{|  +  x,  |  -  .r}.  In  what  follows,  we  rederive  the 

above  table  one  column  at  a  time,  in  order  from  right  to  left,  and  filling  in  each  column 
from  top  to  bottom.  First,  the  desired  spectrum  of  the  final  Gram  matrix  gives  us  that 
^5:5  =  d5;4  =  0  and  /f5;3  =  A5. 2  =  /l5;i  =  |.  Next,  we  wish  to  find  all  {/f4;m}^=1  such 
that  {A4;m}*n=l  c  {A5-,m}5m=l  and  {A4;mfm=1  >  {pm}Am=v  To  this  end,  taking  n  =  5  and  k  =  4, 


Theorem  17  gives: 


max{/l5;5,  /f5;4  +  T5;5  -p5}  <  A4[4  <  min 


min 

/=  1 . 4 


m=l+ 1 


0  =  max{0,  -1}  <  /14;4  <  min{0,  |,  |,  2, 1}  =  0, 
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and  so  d4;4  =  0.  For  each  k  =  3,2, 1,  the  same  approach  gives  d4; 3  =  |,  d4;2  =  f,  and 
d4;i  =  |.  For  the  next  column,  we  take  n  =  4.  Starting  with  k  =  3,  we  have: 

3  3 

max{d4;4,  d4;3  +  /l4;4  -  yu4}  <  /t3;3  <  min  |/t4;3,  ^min  z  ^4;mjk 

m=l  m=l+ 1 

0  =  max{0,  -|}  <  d3;3  <  min{f,  §,  |,  1}  = 

Notice  that  the  lower  and  upper  bounds  on  A3-3  are  not  equal.  Since  T3.3  is  our  first  free 
variable,  we  parametrize  it:  /f3;3  =  x  for  some  x  6  [0,  |].  Next,  k-2  gives: 

|  -  x  =  max{|,  |  -  x)  <  A3-2  <  min{|,  |  -  x,2  -  x}  =  |  -  x, 
and  so  d3;2  =  |  -  x.  Similarly,  /t3;1  =  Next,  we  take  n  =  3  and  k  =  2: 

maxjx,  |}  <  A2j  <  min{|  -  x,  |  +  x,  1}. 

Note  that  A2;2  is  a  free  variable;  we  parametrize  it  as  A2;2  =  y  such  that 

y  €  [f ,  I  +  X]  if  x  e  [0,  |],  y  e  [x,  f  -  x]  if  x  e  [|,  f]. 

Finally,  /l2;i  =2  -y  and  /11;1  =  1. 

It  turns  out  that  when  all  {jum}"=1  are  of  equal  lengths,  the  upper  bound  (4.32)  of 
Theorem  17  can  be  simplified.  In  this  case,  the  majorization  requirement  that  {T„_ v^n}n~J1  > 
{Hm}m=i  comes  for  free.  This  is  a  result  of  the  fact  that  any  nonnegative  nonincreasing 
sequence  {x,„}";=1  of  sum  5  =  XIUi  xm  majorizes  the  uniform  sequence  {wm}"|=1  where 
um  =  ^  for  all  m  =  1 . .  .n.  To  see  this,  first  note  for  k  =  1 ,...,«,  since  {xm}"=1  is  in 
decreasing  order,  the  sum  of  the  first  k  elements  of  this  sequence  will  always  be  greater 
than  or  equal  to  the  sum  of  any  k  elements  chosen  from  {xm}nm=v  Averaging  over  all  m 
k-combinations,  we  have: 

k  i  n  n  n  n 

Zj  X,n  —  7A  Zi  Zi  Zi  ’  ’  ’  Z;  ^Xm  +  xm2  •  •  •  +  xmk).  (4.42) 

m—  1  1^1  mi  =  l  m2=l  m3  =  1  m^=l 

m2^m\  m3^mi,m2  micizm\...mi(-\ 
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Taking  into  account  that  each  xm  is  repeated  a  total  of  f  ”_j)  times,  the  summations  in  (4.42) 
can  be  collapsed  into  one  and  further  simplified  to  give  the  majorization  condition, 


Ct-i)  v  _  ~  1)!  k\(n  -  k)\ 

^  ^ XW  "  n\ 

which  holds  for  k  =  1 It  is  straightforward  to  show  equality  in  the  case  that  k  =  n. 
Thus,  the  uniform  sequence  {«m}"=1  is  indeed  majorized  by  {xn}nm=v  Returning  to  the  upper 
bound  (4.32)  of  Theorem  17  in  the  case  when  {pm}n  =  l  are  all  of  equal  lengths,  we  will  now 
show  that  the  minimum  always  occurs  when  /  =  1.  Hence,  the  new  upper  bound  for  A„-i-k 
can  be  found  by  taking  the  minimum  of  just  two  quantities  rather  than  k  +  1  quantities. 

Theorem  20.  Let  {T„;m}"=1  and  {//„,}"  =1  be  nonnegative  nonincreasing  sequences.  Suppose 
1  \ n-m  =  n/i.  Then  2^=1  An-i;m  =  (n  -  l)ju  and  Un-um^i  E  ttn;m}nm=l  if  and  only  if 
Xn-1-k  e  5„-i;*]/0r  every  k=l9...9n-l,  where 

An-l;k  ,  X^fc  (4.43) 

Bn-i;k  niin Xn-\  (4.44) 


Z 

m=  1 


m=l  m=l  m=l 


where 

n  n-l 

Vn\k  ~  H  ^  ^  /^n;m  ^  ^  Xn—\-m.  (4.45) 

ra=&+l  m=/:+l 

//ere,  we  r/gr/m  w.ve  //ie  convention  that  sums  over  empty  sets  of  indices  are  zero. 
Moreover,  suppose  . . .  ,An-itk+\  are  consecutively  chosen  to  satisfy  these  bounds. 

Then  An_l:k  <  Bn_ ,  .k,  and  so  An-\:k  can  also  be  chosen  from  such  an  interx’al. 

Proof.  The  proof  here  is  similar  to  that  of  Theorem  17.  For  the  sake  of  notational 
simplicity,  we  again  let  {am}'r];=\  :=  {An^.m}n~^v  =i  :=  i An-,m}nm=v  vn-k  =  vk,  Ak  :=  An_ uk, 
and  Bk  :=  uk. 

The  “=>”  direction  of  the  proof  follows  immediately  from  Theorem  17.  For  the  “<=” 
direction,  suppose  Ak  <  ak  <  Bk  for  every  k  -  1 1 .  Then  the  first  entries  in  (4.43) 
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and  (4.44)  give  f3k+]  <  ak  <  pk  for  every  k  =  1, . . .  ,n  -  1,  that  is,  {arm  E  {/?,„}"  =1,  as 
claimed.  It  remains  to  be  shown  that  YIm=\  am  =  (n  -  l)/r.  For  k  =  \,P\-v\  <a\  <  P\-v\ 
and  so 

n  n—  1 

cri  =  /3i  -  Vi  =  ySi  -  |/r  -  ^/3m  +  ^  crmj  (4.46) 

ra=2  m= 2 

Rearranging  (4.46)  and  using  the  fact  that  £”=]  An  =  gives 

Yl—  1  ft 

Zj  Q'm  =  X  A»  “  =  »A'  -  =  0  -  l)y« 

m=l  m=l 

as  claimed.  For  the  final  claim,  we  first  show  that  the  claim  holds  for  k  =  n  -  1,  namely 
that  A„_i  <  Explicitly,  we  need  to  show  that 

max{/3„,/?„_i  +  pn  -  /u}  <  min {p„-up\  +  pn  -  /u}.  (4.47) 

Note  that  (4.47)  is  equivalent  to  the  following  inequalities  holding  simultaneously: 

(i)  pn  <Pn- 1, 

(ii)  Pn—  1  Y-  pn  /J  <  Pn- 1 5 

(iii)  Pn  <p\  +pn  -fi, 

(iv)  Pn- 1  +  Pn  -  n  <  P\  +  Pn  ~  H- 

As  in  the  proof  of  Theorem  17,  (i)  follows  immediately  from  the  fact  that  {/?,„}”  =1  is 
nonincreasing.  For  (ii)  and  (iii),  we  use  the  fact  that  the  average  always  falls  between 
the  maximum  and  minimum  values  of  \Pm\nm=] : 

j  77  1  ^ 

A*  =  min  pm  <  -  V j8m  =  fi  =  -  V  pm  <  max  yS„,  =  /?,.  (4.48) 

m=l  m=l 

Items  (ii)  and  (iii)  follow  immediately  from  (4.48). 

We  now  proceed  by  induction.  Assume  ak+\  satisfies  Ak+ 1  <  ak+\  <  Bk+ 1.  Given  this 
assumption,  we  need  to  show  that  Ak  <  Bk.  Considering  the  definitions  (4.43)  and  (4.44) 
of  Ak  and  Bk,  this  is  equivalent  to  the  following  inequalities  holding  simultaneously: 
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(i)  A+i 


(ii)  pk  -  vk  <  pk, 

(iii)  Pk+i  <P  1  -  v*. 

(iv)  pk  -  vk  <pi-vk 

Again,  the  fact  that  {/?OT}"=1  is  nonincreasing  implies  (i).  Next,  substituting  the  definition  of 
vk+l  from  (4.45)  gives 

n  n- 1 

Vfc+1  -  Pk+ 1  +  «A+1  -  /r  -  ^  Pm  +  ^  Cfin  _  A+l  +  ®/t+l 

m=k+ 2  m=k+2 

n  n- 1 

&+ 1  m=k+ 1 

=  n.  (4.49) 

Next,  note  that  our  inductive  hypothesis  gives  a^+i  >  Afc+1  >  /?/c+ ,  -  v/(+i .  Combining  this 
fact  with  (4.49)  then  gives  vk  >  0,  which  then  implies  (ii).  Next,  substituting  (4.49)  into 
O'k+i  <  Bk+ 1  <  pi  -  vk+l  gives 

0  <  p\  -  ak+i  ~  Vk+\  -  P i  -  ock+i  -  (n  +  pk+\  -  ak+i)  =  Pi  -  vk  -  pk+i, 

which  is  a  rearrangement  of  (iii).  Finally,  (iv)  follows  from  the  fact  that  {/?,„ }”  ,  is 
nonincreasing.  □ 

We  now  give  an  example  to  show  the  necessity  of  the  minimum  over  all  /  in  (4.32). 

Example  21.  Let  N  =  4,  {Ai,A2,Aj,A4}  =  {11,  8,  8, 1}  and  {jUi,//2,//3,//4}  =  {10, 6, 6, 6}.  In 
this  example,  we  show  that  when  {/i,„}{ |j=|  are  not  of  equal  lengths,  the  minimum  does  not 
always  occur  when  1=1. 

Our  goal  is  to  construct  a  sequence  of  inner  eigensteps  {{An-m}’’n=l}l=l  which  satisfy 
Definition  12.  We  will  show  that  every  sequence  of  eigensteps  is  given  by  the  following 
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table: 


n  1 


An:4 

A n:2 

An:i  10 


2  3  4 

1 

x  8 

y  8  8 

16  —y  14  — jc  11 


(4.50) 


where  3  <  x  <  4  and  2  +  x  <  y  <  6. 

We  rederive  this  table  one  column  at  a  time  working  backwards  from  right  to  left. 
To  begin,  the  desired  spectrum  of  the  final  Gram  matrix  completes  the  last  column  of 
(4.50);  that  is,  d4;1  =  11,  A4;2  =  8,  A4; 3  =  8,  and  T4;4  =  1.  Next,  we  complete  the 
third  column  of  (4.50)  and  find  all  {A3-m)li=l  such  that  ]/t3;m}3(=1  E  {d4;m}^=1  and  such  that 
>  {Hm?m=v  Taking  n  =  4  and  k  =  3,  Theorem  17  gives 

ma x{d4;4,  d4;3  +  /l4;4  -  n4)  <  Ay_3  <  min  j/Ly,  ^min^ 

m=l  m=l+l 


^  .  Am  ^  .  A4  ] 


3  =  max{l,3}  <  T3;3  <  min{8,6, 4, 6}  =  4. 


Notice  that  the  lower  and  upper  bounds  on  A^  are  not  equal.  Since  A^-j  is  our  first  free 
variable,  we  parametrize  it:  /I3.3  =  x  for  some  x  6  [3,4].  Also,  notice  that  the  minimum 
occurs  at  /  =  2  which  demonstrates  that  when  {//m}^=1  are  not  of  equal  lengths,  it  is 
necessary  to  consider  the  minimum  over  all  /  in  order  for  {T3;m}^=1  >  {//,„}, 3„=1.  Indeed, 
if  we  had  followed  the  approach  of  Theorem  20,  the  upper  bound  on  x  would  occur  at  /  =  1 
implying  v  e  [3,6].  However,  picking  v  =  6,  leads  to  T3;i  =  8,  contradicting  the  fact  that 
10  =  H11  ^  /I3 ;  1 .  Finally,  to  complete  the  third  column,  we  are  left  with  the  cases  when 
k  =  2  and  k  =  1.  Following  the  approach  of  Theorem  17  gives  A3:2  =  8  and  T3;i  =  14  -  x. 
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For  the  second  column,  we  take  n  =  3.  Starting  with  k-  2,  we  have 

f 

max{A3;3,A3;2  +  A3;3  -  ju3}  <  A2:2  <  min  d3;2,  min 

l  /=  v 

m—l  m=l+ 1 

2  +  x  =  max{.v,  2  +  xj  <  A2:2  <  min{8, 8, 6}  =  6. 

Note  that  /No  is  a  free  variable;  we  parametrize  it  as  A2;2  =  y  such  that  2  +  x  <  y  <  6. 
Finally,  A2-j  =  16  -y  and  d1;1  =  10. 

In  conclusion,  we  now  give  a  complete  constructive  solution  to  Problem  9,  that  is,  the 
problem  of  constructing  every  frame  of  a  given  spectrum  and  set  of  lengths.  Recall  from 
the  beginning  of  this  chapter  that  it  suffices  to  prove  Theorem  10: 

Proof  of  Theorem  10:  We  first  show  why  such  an  F  exists  if  and  only  if  we  have 
m-E.  1  u  v»Lm+ 1  >  {/jn\^=r  Though  this  may  be  quickly  proven  using  the  Schur- 
Horn  Theorem — see  the  discussion  at  the  beginning  of  Section  4.2 — it  also  follows  from 
the  theory  of  this  chapter.  In  particular,  if  such  an  F  exists,  then  Theorem  2  implies 
that  there  exists  a  sequence  of  outer  eigensteps  corresponding  to  {Am}^=l  and  {//„}^=1;  by 
Theorem  14,  this  implies  that  there  exists  a  sequence  of  inner  eigensteps  corresponding 
to  {Am}^=l  U  {0}^=M+1  and  by  the  discussion  at  the  beginning  of  Section  4.3,  we 

necessarily  have  {/!,„( "=|  U  {0}^=M+1  >  \nn\Nn=v  Conversely,  if  {An)%=i  U  {0}^=M+1  >  {n„}*=1, 
then  Top  Kill  (Theorem  16)  constructs  a  corresponding  sequence  of  inner  eigensteps,  and 
so  Theorem  14  implies  there  exists  a  sequence  of  outer  eigensteps  corresponding  to  {Amj^=l 
and  {nn}N=l,  at  which  point  Theorem  2  implies  that  such  an  F  exists. 

For  the  remaining  conclusions,  note  that  in  light  of  Theorem  2,  it  suffices  to  show 
that  every  valid  sequence  of  outer  eigensteps  (Definition  1)  satisfies  the  bounds  of  Step  A 
of  Theorem  10,  and  conversely,  that  every  sequence  constructed  by  Step  A  is  a  valid 
sequence  of  outer  eigensteps.  Both  of  these  facts  follow  from  the  same  two  results.  The 
first  is  Theorem  14,  which  establishes  a  correspondence  between  every  valid  sequence 
of  outer  eigensteps  for  {Am}^=1  and  {pn}^=1  with  a  valid  sequence  of  inner  eigensteps 


/4;i  ^  ^  ^3;n 
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for  {Am}%=1  u  {0}^=M+1  and  {/i„}^=1  and  vice  versa,  the  two  being  zero-padded  versions 
of  each  other.  The  second  relevant  result  is  Corollary  18  which  characterizes  all  such 
inner  eigensteps  in  terms  of  the  bounds  (4.31)  and  (4.32)  of  Theorem  17.  In  short,  the 
algorithm  of  Step  A  is  the  outer  eigenstep  version  of  the  application  of  Corollary  18  to 
|  Am}^=  |  U  !0}')'=  w+| ;  one  may  easily  verify  that  all  discrepancies  between  the  statement  of 
Theorem  10  and  Corollary  18  are  the  result  of  the  zero-padding  that  occurs  in  the  transition 
from  inner  to  outer  eigensteps. 
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V.  Completing  finite  frames 


In  this  chapter,  we  generalize  the  theory  of  Chapters  3  and  4  and  address  the  problem 
of  completing  finite  frames  for  a  given  nontrivial  initial  spectrum.  We  wish  to  complete  the 
frame  FN  =  !  /„}')!,  by  adding  P  additional  measurements  in  order  to  construct  F  = 
whose  frame  operator  has  spectrum  {A ,„}^=1.  Our  goal  is  to  characterize  all  such  frames 
whose  final  spectrum  is  (a,  p)-constructible: 

Definition  22.  Given  nonnegative  nonincreasing  sequences  a  =  {am}^=1  and  p  =  {fn+p}p  v 
a  corresponding  nonnegative  nonincreasing  sequence  {Am}^=l  is  (a,  p)-constructible  if 
given  any  N  vectors  Fn  =  \.fn)Nn=x  in  CM  whose  frame  operator  FNF*N  =  Zn=i  fnfn  has 
spectrum  {am}^=l,  it  is  possible  to  find  vectors  {fN+p}pp=l  such  that  ||/'v+/)||2  =  /uN+p  for  all 
p  =  1, . . .  P,  and  such  that  the  new  frame  operator  F  =  has  spectrum  {Am}%=l . 

In  particular,  in  this  chapter  we  solve  the  following  problem: 

Problem  23.  Given  nonnegative  nonincreasing  sequences  a  =  {am}^=i  and p  =  {pN+p}pp=v 
characterize  all  (a,  p)-constructible  sequences  {dm|^=|. 

Specifically,  the  major  results  of  this  chapter  are  Theorem  39  which  characterizes 
which  spectra  are  (a,yu)-constructible  and  Theorem  40  which  provides  the  explicit  Chop 
Kill  algorithm  for  constructing  a  sequence  of  continued  outer  eigensteps. 

To  solve  Problem  23,  we  build  on  the  theory  of  Chapters  3  and  4  to  characterize  all 
frames  whose  final  spectrum  is  (a,yu)-constructible.  Taken  together,  the  results  of  these 
two  previous  chapters  allow  us  to  explicitly  parametrize  the  set  of  all  frames  whose  frame 
operator  FF*  has  a  given  spectrum  and  whose  elements  have  a  given  set  of  lengths  in  the 
case  where  no  initial  spectrum  is  given,  i.e.,  where  am  =  0  for  all  m.  In  particular,  we 
saw  that  any  F  for  which  FF*  has  {Am}^=1  as  its  spectrum  and  for  which  ||  /„||2  =  pn  for 
all  n  generates  a  sequence  of  eigensteps.  The  main  result  of  Chapter  3,  Theorem  2,  proves 
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that  the  converse  of  this  statement  is  also  true  and  characterizes  and  proves  the  existence 
of  sequences  of  vectors  that  generate  a  given  sequence  of  eigensteps.  In  light  of  these 
facts,  solving  Problem  23  can  be  reduced  to  finding  a  valid  sequence  of  continued  outer 
eigensteps  which  have  the  additional  property  that  AN;m  =  am: 

Definition  24.  Given  nonnegative  nonincreasing  sequences  {am\^=],  {Am}^=]  and  {pN+p}pp=v 
a  sequence  of  continued  outer  eigensteps  is  a  doubly-indexed  sequence  of  sequences 
{{AN+P-m}^=l}p=Q  for  which: 

(i)  The  initial  sequence  is  prescribed:  AN-m  =  am,  dm  =  1, . . . ,  M. 

(ii)  The  final  sequence  is  {d,„}^=1:  AN+P-m  =  Am,  dm  =  1 

(iii)  The  sequences  interlace:  {AN+P-  i;m}“=1  E  {AN+p-m}™=v  dp  = 

(iv)  The  trace  condition  is  satisfied:  YZ=  i  ^N+P-,m  =  i  am  +  Zp,=1  Pn+P',^P  =  1 

The  differences  between  Definition  24  and  Definition  1  of  outer  eigensteps  in 
Chapter  3  are  conditions  (i)  and  (iv).  Conditions  (i)  and  (iv)  are  both  results  of  the  fact 
that  we  are  adding  measurements  to  the  existing  spectrum  a  =  {am}ffI=v  In  this  chapter,  we 
are  not  concerned  with  constructing  eigensteps  for  a  as  this  problem  has  been  completely 
resolved  in  Chapter  4  which  provides  an  explicit  parametrization  of  the  set  of  all  valid 
eigensteps  for  a.  Instead,  we  focus  our  attention  on  constructing  the  continued  outer 
eigensteps  for  {Am}^=l .  To  this  end,  our  goal  is  to  complete  the  following  table  of  continued 
outer  eigensteps  for  given  initial  and  final  spectra: 

N  +  p  0  1  2  ...  P 

d-N+p-M  <*3  ?  ?  •  •  • 

d-N+p-\  ?  ?  ••• 
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Note  that  once  we  have  a  sequence  of  continued  outer  eigensteps,  we  can  then  apply  the 
algorithm  of  Theorem  7  in  order  to  complete  the  frame  FN  by  explicitly  constructing  the 
added  frame  vectors,  \  fN+p\pp=v 

In  order  to  solve  Problem  23  we  first  consider  a  class  of  sequences  {Am}^1=l  which  is 
larger  (and  easier  to  understand)  than  the  (a,/r)-constructible  class  that  we  are  interested 
in.  If  a  -  {am\M=]  is  the  initial  sequence  of  eigenvalues,  we  know  from  the  interlacing 
condition  of  Definition  24  that  am  =  AN-m  <  AN+p;m  for  p  =  1  Since  a  is  nonnegative, 

{Aiv+p;m}M=  i  is  nonnegative  as  well.  Sequences  which  have  these  properties  are  what  we  call 
a-admissible : 

Definition  25.  Given  a  nonnegative  nonincreasing  sequence  a  =  {a„ ,}^=1,  a  corresponding 
nonincreasing  sequence  {Tm}^=1  is  a-admissible  if  am  <  Am,  for  all  m  =  1, . . . ,  M.  The  set 
of  all  a-admissible  sequences  is  denoted  by  adm(a). 

Every  (a,//)-constructible  sequence  is  an  a-admissible  sequence,  but  the  converse  is 
not  necessarily  true.  Due  to  the  complexity  of  Problem  23  we  first  understand  what  it 
means  for  a  sequence  to  be  a-admissible.  Once  we  have  restricted  our  search  to  only 
those  sequences  which  are  a-admissible,  we  then  tackle  the  problem  of  determining  what  it 
means  for  one  of  these  sequences  to  be  (a,/r)-constructible.  To  begin,  in  the  next  section  we 
consider  how  only  a  single  eigenvalue  is  related  to  a  given  initial  sequence  of  eigenvalues 
a.  In  Section  5.2  we  then  turn  our  attention  to  solving  Problem  23.  There  we  make  use  of 
the  theory  in  Section  5.1  to  create  a  one-to-one  correspondence  between  an  a-admissible 
sequence  and  a  new  matrix  which  represents  the  relationship  between  the  initial  sequence 
a  and  final  sequence  {Am}^=v  We  also  characterize  sequences  {Am}^=]  which  are  (a,/i)- 
constructible  and  generalize  the  Top  Kill  algorithm  of  Chapter  4  for  generating  a  valid 
sequence  of  continued  outer  eigensteps. 


89 


5.1  Preliminaries 


In  this  section  we  explore  the  connection  between  a  single  eigenvalue  A  and  a  given 
initial  sequence  of  eigenvalues  a  -  {am}^=l.  The  following  two  functions  will  help  us  relate 
A  to  a: 

Definition  26.  Let  p  e  Mm+1.  Define  the  function  s  :  Mm+1  — >  M 

M+ 1 

s<p):=2>-  (5.1) 

71=1 

Definition  27.  Let  A  e  [0,  oo).  Given  a  nonnegative  nonincreasing  sequence  a  =  {arm}£f=1, 
define  pff  :  [0,  oo)  — »  MM+1 


[pff(d)]„  :=  t{[Q,  A]  n  (a„, (5.2) 

where  £  is  the  length  (Lebesgue  measure)  of  an  interval  and  we  adopt  the  convention  that 
c?o  :=  oo  and  aM+  i  :=  0. 

The  function  s  simply  computes  the  sum  of  the  elements  of  any  vector  whereas  p„  can 
be  thought  of  as  taking  any  nonnegative  real  number.  A,  and  breaking  it  up  into  intervals 
according  to  a.  In  other  words,  p„  partitions  A  according  to  a  leading  to  the  following 
definition. 


Definition  28.  Define  the  set  of  all  a-pcirtitions  as  follows: 

0  <pn  <  an- 1  -cr„,Vn; 


Part(ar)  :=  \p  6 


SM+ 1 


(5.3) 


if  pn  >  0,  then  pn>  =  an>-\  -  an>,  'in'  >  n 

In  a  moment,  we  will  show  that  Part  (o')  is  the  range  of  pff.  The  next  result  states  that 
any  vector  in  Part(o')  must  be  of  a  particular  form. 
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Lemma  29.  Given  a  nonnegative  nonincreasing  sequence  a  =  {«m)“=1  and  p  e  Part(or), 
there  exists  an  index  n{]  -  \  ...  M  +  \  such  that: 


0, 

ifn  <  n0. 

s(p)  -  an. 

if  n  =  no. 

(5.4) 

^71-1  ^71? 

if  n  >  n0. 

Specifically,  for  nonzero  p  e  Part  (o'),  n0  is  the  unique  index  such  that  ano  <  s(p)  <  a„0-i. 

Proof.  Let  p  =  {p,,}^1  e  Part(o').  We  claim  that  in  the  case  where  pn  =  0  for  all 
n  =  1,...,M  +  1,  we  can  choose  n0  =  M  +  1.  Indeed,  we  immediately  have  p„  =  0 
for  n  <  M+l.  Moreover,  for  n  =  M  +  l,  pn  =  s(p)  -  or„  =  s(0)  -  q-m+i  =  0  since  by 
definition,  aM+ 1  :=  0.  Hence  p  =  0  is  of  form  (5.4). 

Now  consider  the  case  where  p  4=  0.  Pick  n0  to  be  the  minimal  index  such  that  p„0  >  0. 
Since  no  is  the  minimal  such  index,  pn  =  0  for  all  n  <  n0.  Moreover,  by  definition  of 
Part(o'),  pn  =  <+„_  |  -  an  for  all  n  >  n{).  In  light  of  these  facts, 

M+ 1  no- 1  M+l 

Pm]  —  y  ^  Pn — y  ^  Pn  —  y  ^ 

71=1  71=1  71=710  +  1 

M+l 

=  S(p)  -  0  -  (°+l  -  »n) 

71=710  +  1 

=  s(p)  -  (ano  -  %+i) 

=  s  (p)  -  a„0, 

where  the  final  equality  follows  from  the  fact  that  aM+i  :=  0.  Thus,  p  is  of  form  (5.4). 

Next  we  show  that  the  index  n0  is  unique  provided  p  4=  0.  Take  any  n0  such  that  (5.4) 
holds.  First  note  that  since  s(p)  -  a„0  =  p„0  >  0,  this  gives  that  cr„0  <  s(p).  Moreover,  since 
pn  =  0  for  all  n  <  n{)  and  p„  <  (an-\  -  an )  for  all  n  by  (5.3),  we  have: 

M+l  M+l  M+l 

S(P)  —  ^  ^  Pn  —  y  ' j  Pn  —  ^  ^(o+i-l  ®n)  —  Q+o-1  ®M+1  —  I  • 

71=1  71=7lQ  71=710 
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Putting  these  two  facts  together  yields  ano  <  s ip)  <  cr„0_i.  This  implies  that  the  index  n{)  is 
unique  since  such  an  inequality  can  only  hold  for  at  most  one  n0.  □ 

In  the  next  section,  we  will  see  that  solving  Problem  23  will  require  a  one-to-one 
correspondence  between  any  eigenvalue  Am  and  its  cr-partition  p,, (/{,„).  The  following 
theorem  states  that  p„  is  a  bijection  from  the  nonnegative  real  numbers  onto  their 
corresponding  or-partitions. 

Theorem  30.  pa  is  a  bijection  from  [0,  oo)  onto  Part(cr),  with 

Pa-1  =  s  :  Part(a)  ->  [0,  oo). 


Proof.  We  first  show  that  s  is  a  left-inverse  of  p„  for  any  A  >  0: 

M+ 1  M+l 

s(pa(d))  =  ^][pa(T)]„  =  ^([0,d]  n  (ar„a„m) 

71=1  71=1 

/M+l 

=  n  (an,  an-\\ 

''72=1 

=  e[\0 ,A]  n  | 


(5.5) 


By  definition,  a0  :  =  oo  and  aM+i  :=  0,  implying  U^li  (an,  an-i\  =  (0,  oo].  Coupled  with 
the  fact  that  A  >  0,  (5.5)  becomes 


s(pff(T))  =  £([0,  A ]  n  (0,  oo])  =  £((0,  A])  =  A. 


Thus,  s  is  indeed  a  left-inverse  of  p„,  in  particular  implying  p„  is  one-to-one. 

To  show  that  s  is  a  right-inverse  of  Pq.,  let  p  =  {pn}\ IV  e  Part(a),  and  without  loss  of 
generality,  assume  p  ±  0.  Then  by  Definition  27, 


0,  if  s (p)  <  an. 


[p^sfp))]*  =  ^([0,  s(p)]  n  («„,«„_ i])  =  ^ 


s  (p)  -  an, 


if  an  <  s (p)  <  a„-\. 


(5.6) 


Otn—  1  &n.  If  Qtn—1  ^  S(p). 
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Moreover,  since  p  ^  0,  the  n{)  of  Lemma  29  is  the  unique  index  such  that  a„0  <  s(p)  <  or„0_i. 
We  use  this  fact  to  prove  that  pff(s(p))  =  p  by  relating  (5.6)  to  (5.4).  That  is,  we  want  to 
show  that 

0  if  s (p)  <  a„,  0  if  n  <  nQ 

'  s(p)  -  an,  if  an  <  s(p)  <  an- 1,  =  '  s(p)  -  a„,  if  n  =  n0,  ’  ■  (5-7) 

(xn—i  if  Q^n—\  ^  s(p).  if  n  ^  Hq. 

In  particular,  considering  (5.7)  for  an  index  n  for  which  s(p)  <  an,  (5.6)  gives  [p„(s(p))]„  = 

0.  Moreover,  since  ano  <  s(p)  <  an  and  {am}^=l  is  nonincreasing,  n  <  n{)  and  so  (5.4)  gives 
p„  =  0.  Meanwhile,  for  n  such  that  s(p)  >  ar„_i,  (5.6)  gives  [po.(s(p))]„  =  ar„_i  -  a„  while 
(5.4)  gives  p„  =  an-\  -  a„  since  the  fact  that  an-\  <  s(p)  <  ar„0_i  implies  n  >  n0.  In  the  final 
case  where  n  happens  to  satisfy  an  <  s(p)  <  cr„_i,  we  necessarily  have  that  n  -  n{) — the 
index  «0  in  Lemma  29  is  unique  when  p  <  0 — implying  [pff(s(p))]„  =  s(p)  -  a„0  =  p„. 

Having  that  s  :  [0,  oo)  — >  Part(a)  is  both  a  left  and  right  inverse  of  p„,  then  we  have  pff  is  a 

bijection  from  [0,  oo)  onto  Part(a).  □ 

The  following  example  demonstrates  how  we  use  the  function  piy  to  go  back  and  forth 
between  a  single  eigenvalue  A  and  its  a-partition. 

Example  31.  Let  M  =  3,  A  =  ^  and  {aq,  a2,  a3}  =  {\,\,  ^}.  The  cr-partition  of  A,  pff(/l),  is 
the  1x4  vector  where  each  entry  in  computed  using  Definition  27.  That  is, 

[Po-(4)]„  =  £([0,T]  n  (a,,,®,,.!]). 

When  77  =  1,  by  definition,  ar0  :=  oo  and  so 

[p«(4)]i  =  mo,  A]  n  (aua0D  =  €([0,  f  ]  n  (J,  oo])  =  £((},  f  ])  =  §.  (5.8) 

Continuing  for  n  =  2  and  n  =  3, 

[p  aWh  =  mo  ,a]  n  (a2,am  =  mo,  f  ]  n  (|,  id  =  m\,  id  =  1, 

[Pa(4)]3  =  mo ,A]  n  (a3,a2])  =  A[0,  f  ]  n  (±,  |])  =  €{{\,  ;])  =  (5-9) 
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Finally  in  the  case  that  n  =  4,  we  use  the  fact  that  a  4  :=  0  which  gives 


[p =  mo,  A]  n  (a5,a4])  =  mo,  n  (0,  ±])  =  £((0,  ±])  =  ±  (5.10) 


Taking  (5.8),  (5.9),  and  (5.10)  together  gives  that  the  a  partition  of  A  =  is 


[p  *(¥)] 


3  i  1  1 
2  1  4  2 


(5.11) 


It  is  straightforward  to  verify  that  (5.11)  is  also  consistent  with  Lemma  29  when  na  -  1. 


In  the  next  section,  we  use  these  results  to  relate  the  initial  spectrum  {am}^=1  to  the 


1 M 


final  one,  {Am,m=v 


5.2  (a,  //)-constructible  sequences  and  Chop  Kill 

In  this  section,  we  focus  on  Problem  23,  namely  the  problem  of  finding  all  ( a,/u )- 
constructible  sequences.  Solving  this  problem  involves  finding  a  valid  sequence  of 
continued  outer  eigensteps  (Definition  24)  for  given  nonnegative  nonincreasing  sequences 
(4)Jf=1,  a  =  { am}^=  i  and  F  =  {/r,v+p(p=1-  Just  as  in  Chapter  4,  we  can  view  the  problem 
of  constructing  a  sequence  of  continued  outer  eigensteps  as  building  a  staircase — the 
difference  here  being  that  we  already  have  an  initial  sequence  of  eigenvalues,  or  stairsteps, 
to  build  on. 

When  no  initial  spectrum  is  given,  we  have  already  seen  that  the  highest  levels  are 
the  hardest  the  build.  This  motivated  the  Top  Kill  algorithm  for  constructing  a  sequence 
of  inner  eigensteps  by  removing  as  much  area  from  the  top  of  the  stairsteps  as  possible 
since  these  areas  require  the  most  foresight  to  build.  In  that  case,  a  sequence  of  inner 
eigensteps  existed  if  and  only  if  {d„}^=|  majorized  {//„};)(,  where  {d„}^=1,  the  spectrum  of 
F*F,  is  a  zero-padded  version  of  {Am)^i=]  when  M  <  N.  In  contrast,  when  we  consider 
the  current  problem  of  constructing  continued  outer  eigensteps  on  top  of  a  given  nontrivial 
initial  spectrum,  the  areas  that  are  the  hardest  to  build  are  those  that  are  the  highest  relative 
to  the  initial  spectrum  a.  In  this  setting,  determining  whether  a  sequence  of  eigensteps 
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exists  is  not  as  straightforward.  In  this  section,  we  solve  Problem  23  by  introducing  a 
new  algorithm,  the  Chop  Kill  algorithm,  for  generating  a  valid  sequence  of  continued  outer 
eigensteps  when  starting  from  a  nontrivial  initial  spectrum  a.  In  order  to  solve  this  problem 
we  build  on  the  theory  of  Section  5.1  in  order  to  relate  the  initial  spectrum  a  to  the  final 
spectrum  {Am\^=v  First,  consider  the  following  definitions  which  are  vectorized  version  of 
Definitions  26  and  27  in  Section  5.1. 

Definition  32.  Let  A  e  MMx(M+i).  Define  the  function  S  :  ]RMx(M+1)  — >  Mm, 

[S(A)]m  :=  s(Am,:).  (5.12) 

Definition  33.  Given  a  nonnegative  nonincreasing  sequence  a  =  {am}^=l  let  {Am}^=1  be 
a-admissible.  Define  the  function  P,t.  :  adm(cr)  — >  RMx(M+1), 

[P«(UmC=l)]m,«  :=  [p MJln  =  ^([0,  Am]  Pi  (a„,  CT„_i]),  (5.13) 

where  p„  is  the  function  defined  in  Definition  27. 

The  function  S  operates  by  applying  s  defined  in  Section  5.1  to  each  row  of  the  matrix 
A.  That  is,  S  sums  across  rows.  Similarly,  P„  operates  by  applying  p„  to  each  element  of 
{Am}M=l  and  then  stacking  the  outputs  as  rows  of  some  M  x  (M  +1)  matrix  A.  That  is, 
the  mth  row  of  Pa({/lm)^=])  represents  how  the  eigenvalue  Am  is  broken  up  into  intervals 
according  to  the  initial  spectrum  a.  Because  of  this,  we  refer  to  A  =  Pa({Am}^=1)  as  the 
spectral  partition  matrix  of  {Am}M=v  The  following  example  demonstrates  how  to  find  the 
spectral  partition  matrix  for  a  set  of  eigenvalues. 

Example  34.  Let  M  =  3.  We  now  use  Definition  33  to  construct  the  3x4  spectral  partition 
matrix  Pa({Tm}^=1)  where  {Ti,T2,/i3}  =  {-j,  1},  and  {au  a2,  <d}  =  {},  |}-  As  already 

mentioned,  P(,  operates  by  applying  p„  to  each  element  of  {A,n}zm=x  and  then  stacking  the 
outputs  as  rows  of  the  matrix  Pa({Am}zm=l).  As  such,  we  calculate  the  entries  of  Pff({/tm}^=1) 
one  row  at  a  time. 
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When  m  =  1,  the  entries  of  the  first  row  of  Pa({/im}3  =1)  correspond  to  the  o-partition 
of  Ai  =  -j.  Using  the  result  of  Example  31  where  we  calculated  the  cr-partition  for  A  =  -j, 
the  first  row  of  Pa({dm}3  =1)  is  given  by  (5.11): 


[Patt'ULl)]!,: 


1  1 
4  2 


(5.14) 


Next,  for  m  =  2,  by  Definition  33,  each  entry  of  the  second  row  of  P0,({/fm)^=|)  is  given  by 


[^a(Um?m=i)hn  =  mO,  A2]  n  (an,  an- 1]),  (5.15) 


where  n  =  1, . . . ,  4.  Substituting  n  =  1, . . . ,  4,  into  (5.15)  and  using  the  fact  the  :=  °° 
and  a4  :=  0,  gives 


[Pa(Rm}Li)]2,i  =  £([0,  f  ]  n  (2,  OO])  =  €((l  I ])  =  1, 

[p«({4miLi)]2,2  =  mo,  f]  n  (|,  j])  =  ml  id  =  i, 

[Pa({4m}Ll)]2,3  =  €([0,  f]  n  (i,  §])  =  m\,  |])  =  l 

[Pa({4m}Ll)]2,4  =  €([0,  f]  n  (0,  *])  =  mo,  =  i 

Collectively,  the  equations  of  (5.16)  give  the  second  row  of  P a({dm}3=1): 


[Pa({4m}Ll)]2,:  = 


1 

2 


1 

2 


Repeating  the  calculations  for  d3  =  1,  the  last  row  of  P«({/lm}3=1)  becomes 


[P(,({/ULl)]3,: 


1  1 
4  4 


1 

2 


(5.16) 


(5.17) 


(5.18) 


Stacking  (5.14),  (5.17),  and  (5.18)  together,  the  spectral  partition  matrix  for  {Ai,  A2,  d3}  = 
{f,|,l}is  given  by: 


Mnt 


)m=  l) 


1  1 

4  2 

1  1 
4  2 


0 


1  1 
4  2 


(5.19) 
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Note  that  by  definition,  each  row  of  P a({fim},M=1)  is  an  a-partition.  We  will  show  in 
a  moment  that  a  one-to-one  correspondence  exists  between  any  a-admissible  sequence 
(Definition  25)  and  the  spectral  partition  matrix  it  generates,  Pa({/lm}^=|).  Since  we  will 
show  that  we  are  able  to  go  back  and  forth  between  the  sequence  {Amj^=l  and  its  spectral 
partition  matrix,  P a({Am}^=1),  we  use  the  same  term  a-admissible  to  refer  to  a  specific  class 
of  matrices  as  well: 

Definition  35.  Given  a  nonnegative  nonincreasing  sequence  a  =  {am}„=1  the  M  x  (M  +  1) 
matrix  A  is  a-admissible  if: 


(i)  each  row  of  A  is  an  a-partition,  i.e.,  lies  in  the  set  Part(a)  defined  in  (5.3). 

(ii)  the  values  in  any  column  of  A  appear  in  decreasing  order,  i.e.,  Amn  <  A m_1;„  for  all 
m  =  2  . . .  M  and  n  =  1, . . . ,  M  +  1 . 

(iii)  Am  n  =  <r„_|  -  an  whenever  n  >  m. 

The  set  of  all  a-admissible  matrices  is  denoted  by  Adm(a). 

We  now  prove  that  P,t  is  a  bijection  from  adm(a)  onto  Adm(a). 

Theorem  36.  P„  is  a  bijection  from  adm(a)  onto  Adm(a)  with 

Pa_1  =  S  :  Adm(a)  — >  adm(a). 

Proof.  Let  a  =  {am}M=1  be  nonnegative  and  nonincreasing.  We  first  show  that  P,t  is  well- 
defined,  namely  that  if  {Tm}^=l  e  adm(a)  then  P tt({4(“=1)  G  Adm(a).  First  note  that  since 
Part(a)  is  defined  to  be  the  range  of  Pq,,  each  row  of  Pff({/tm}^f=1)  is  an  a-partition  and  so  (i) 
of  Definition  35  is  satisfied.  For  (ii),  since  {Am}^=1  is  nonincreasing,  [0,  A,n\  c  [0,  for 
all  m  =  2, . . .  M,  which  in  turn  implies  that 

[P u-(RiC=1  )]„!,„  =  f([0, /!,„]  n  (a„,  a„_i])  <  A [(),/{,„_]  ]  n  (a„,a„_i])  =  [Pa({dm}^=1)]m-i,„ 
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for  all  n  =  2, . . . ,  M  +  1.  Finally  for  (iii),  for  any  n  >  m,  we  have  that  n—  1  >  m  which  gives 
Am  >  A„- 1  >  an-i,  where  the  last  inequality  follows  from  the  fact  that  {Am}^=j  e  adm(a).  In 
this  case,  (5.13)  becomes 

[ P+ ( i A/ii }/;;  i ) \ni.n  —  mO,Am]  FI  (o',, ,  (y n-  1 1 )  —  ({{(y n ,  (yn~ \  J )  —  cr«-i  ctn. 

Hence,  Pa({/1,„(M=|)  6  Adm(o')  and  so  Pa  is  into  Adm(cr).  Next  we  show  that  S  is  well- 
defined,  namely  that  if  A  6  Admin)  then  S(A)  6  admin).  Here,  for  any  A  6  Admin),  using 
the  fact  that  Am  n  <  Am_i>n  from  Definition  35. (ii), 

M+l  M+l 

[S(A)],„  —  ^  .  Amn  <  ^  .  A m—iji  —  [S(A)]m_j, 

71=1  71=1 

which  implies  {S(A)m}^=1  is  nonincreasing.  Additionally,  from  property  (iii)  of  Defini¬ 
tion  35, 

M+l  M+l  M+l 

[S(A)]m  =  Z  ^m,n  —  ^  ^  -^771,71  —  I  (®«-l  ®h)  —  Q+n  ^M+l  —  (%m- 

n  I  n=m+l  n=m+ 1 

Putting  together  the  fact  that  {S(A)m}^=1  is  nonincreasing  with  the  fact  that  [S(A)]m  >  nm 
for  all  m  =  1, . . . ,  M,  we  have  {S(A)„,}^=1  is  a- admissible.  Moreover,  we  claim  that  P,t  and 
S  are  inverses  of  each  other  over  these  domains.  This  follows  from  the  fact  that  P„  and  S 
are  simply  vectorized  versions  of  p(>  and  s,  which  were  proved  to  be  inverses  of  each  other 
in  Theorem  36.  □ 

Now  that  we  have  a  one-to-one  correspondence  between  any  sequence  of  eigenvalues 
{Amim=1,  and  its  spectral  partition  matrix  Pa({Am}^=1),  we  simplify  notation  by  denoting 
this  matrix  as  simply  A  for  the  remainder  of  the  chapter.  Now  we  turn  our  attention  to 
Problem  23,  specifically  characterizing  all  (cr,//)-constructible  sequences.  Later  on,  we 
will  see  that  the  lower  diagonals  of  A  play  an  important  role  in  determining  whether  or  not 
a  sequence  {Amj^=l  is  (ar,//)-constructible.  For  each  pair  of  indices  in  M.  -  {( m,n )  :  m  = 
1  =  1, . . . ,  m },  we  now  introduce  a  new  indexing  which  indexes  each  of  these 
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lower  triangular  elements  according  to  the  diagonal  it  belongs  to  as  well  as  its  location  on 
the  diagonal.  In  this  case,  we  use  the  subscript  A where  j  =  1, . . . ,  M  is  for  the  diagonal 
and  k=l,...,M-j+lis  the  location  along  that  diagonal.  We  let 

be  the  set  of  all  pairs  of  indices  in  the  lower  triangle  of  A.  Given  A (r^k),  we  can  go  back 
and  forth  between  its  corresponding  row  and  column  index  (m,  n)  e  At  as  a  function  of  j 
and  k, 

(m,  n)  =  <x(j,  k)  =  (j  -  1  +  k,  k).  (5.20) 

Conversely,  given  A„v,  we  can  determine  its  lower  diagonal  and  location  index  (  j,  k)  e  27 
as  a  function  of  m  and  n. 


(j,  k )  =  r(m,  n)  -  (in  -  n  +  1,  n). 


(5.21) 


For  example,  if  M  =  3,  indexing  the  3x4  matrix  A  according  to  standard  row  and 
column  indexing,  the  (m,  n)  entry  of  A  is  assigned  the  label 


(5.22) 


It  it  straightforward  to  show  that  the  coordinate  transforms  (5.20)  and  (5.21)  are  inverse 
bijections  from  M  to  J~.  We  now  define  the  following  function  which  computes  the  sum 
of  any  lower  diagonal  of  A: 


(U) 

* 

* 

* 

oihi) 

* 

* 

* 

(2,1) 

(2,2) 

* 

* 

= 

cr(2,l) 

oi  1,2) 

* 

* 

(3,1) 

(3,2) 

(3,3)  * 

cr(3,l) 

cr(2,2) 

o-(l,3)  * 

Definition  37.  Let  A  6  MMx,m+i(  Define  the  diagonal  sum  function  DS  :  KMx(M+1)  — »  M'w 

(5.23) 


M-j+ 1  M-j+ 1 

DS(A )j  :=  Z  A cr(j,k)  -  ^  Aj-l+k,k- 


k= 1  k= 1 

The  function  DS  computes  sums  along  the  diagonals  of  the  lower  triangular  part  of 
A  G  For  any  A,  we  claim  that  DS(A)  is  a  nonnegative  nonincreasing  sequence. 
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Indeed,  DS(A)  is  nonnegative  as  a  result  of  Property  (i)  of  Definition  35  which  requires  each 
row  of  A  to  be  an  a-partition,  meaning  all  entries  of  A  are  nonnegative.  Meanwhile,  the 
fact  that  DS(A)  is  nonincreasing  follows  from  property  (ii),  particularly  that  I\m  n  <  Am_i;„. 
Specifically,  for  ;  =  2,...,Mwe  have, 

M-j+l  Af-O'-l) 

[DS(A)]  j  =  X  hj-\+k,k  <  ^  \j-i)-i+k,k  <  [DS(A)]7_J. 

k=  1  k= 1 

The  following  example  demonstrates  how  to  calculate  the  diagonal  sums  of  the  spectral 
partition  matrix  (5.19)  of  Example  34. 

Example  38.  Recall  from  Example  34,  the  spectral  partition  matrix  for  {^i ,  /l2,  -T3}  = 
,  f,  1}  and  {a1,a2,a3}  =  {},  |,  \}\ 


We  now  use  Definition  37  to  calculate  the  lower  diagonal  sums  of  A.  There  are  three  lower 
diagonals  of  A:  DS(A)i  consists  of  three  elements,  DS(A)2  consists  of  two  elements,  and 

DS(A)3  consists  of  only  one  element.  When  j  =  1,  (5.23)  gives 

3  3 

DS(A)i  =  ^  A0.(1)t)  =  ^  A k>k  -  Aj j  +  A2i2  +  A33  =  |  +  1  +  |  =  X’ 

k= 1  k= 1 

Similarly,  when  j  =  2,  (5.23)  gives 

2  3 

DS(A)2  =  ^  Ao-(2>Jt)  =  ^  Ai  +k'k  =  A2-i  +  A3,2  =  \  +  \  =  |. 

£=1  A=1 

Finally,  when  j  =  3,  DS(A)3  only  consists  of  one  element,  and  so  DS(A3)  =  A^j)  =  A31  = 
0.  Notice  that  DS(A)  =  {^,  |,  0}  is  a  nonnegative  nonincreasing  sequence  as  claimed. 

As  previously  mentioned,  the  lower  diagonals  of  A  play  an  important  role  in 
determining  whether  or  not  a  sequence  {Am}^=1  is  (cr,//)-constructible.  In  fact,  we  now 
show  that  if  a  sequence  {Am}^=l  is  (o',yu)-constructible  {DS(A)7}“1  necessarily  majorizes 

{ fJ-N+p } p=  j  • 
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Theorem  39.  Given  nonnegative  nonincreasing  sequences  {am}^=1,  {pN+P}p=i,  and  {Am}^=1, 
which  satisfies  Am  >  am  for  all  m  =  1  if  {Am}ff1=l  is  ( a,p)-constructible  then 

{DS(A);-}f=1  >  [pN+p}pp=v 


Proof  Let  a  =  !  a,,, \^t= , ,  p  -  {Pn+p)p=i  be  nonnegative  nonincreasing  sequences  and 
! Am}^=  |  be  cr-admissible.  If  {Am}^=1  is  (o',yu)-constructible,  there  exists  a  sequence  of 
continued  outer  eigensteps  {{AN+p;m}^=1jp=0  and  a  corresponding  sequence  { A;V+/)}^=0  of 
a- admissible  matrices  where  AN+P  =  Pa({AN+p-m}^l=l)  is  the  spectral  partition  matrix  of 
and  A  :=  AN+P.  We  claim  that 


P  M 


Z  dN+P'  ~  Z  DS(A);, 

(5.24) 

P'= 1  7=1 

/?  min{M,/?} 

Z^v+p'  <  Z  DS(A)b 

P'=l  7=1 

(5.25) 

for  all  p  =  1, ....  P.  From  the  trace  condition  (iii)  of  continued  outer  eigensteps  and  the 
convention  that  aM+\  -  0,  we  have 


M 


M 


P'  =  1 


J]pn+P'  -YjAn+pw-Yj 

m— 1  m—  1 

M  / 

/  J  l/^7V+/?;ra  &M+ 1) 

M+l 

N+p;m  ^  ^  (fln- 1  &n) 


m—  1 
M 


(5.26) 


=2> 


m—  1 


n=m+l 


Since  {AN+p-m}^l={  is  a-admissible  for  any  p.  Theorem  36  gives  that  AN+p  6  Adm(ar). 
Property  (iii)  of  Definition  35  then  gives  that  AN+p-„h„  =  an-\  -  an  whenever  n  >  m.  As 
such,  (5.26)  can  be  rewritten  as 


M 


//  =  ! 


m—  1 


M+l 


z  pn+P'  -  Z  Uw+p 


N+p;m  V N+p\m,n 

n=m+ 1 

M+l 


(5.27) 


Combining  this  with  the  fact  that  AN+p-„,  =  X„=i  AN+p;mM,  (5.27)  becomes 


M  /  M+l 


M+l 


YjP^p'  -  Z  Z  Av+/);'"-"  ■  z  A  N+P  \m,n  A  N+P;,i 

p'-l  m=  1  '  n—l 


n-m+l 


M  m 


(5.28) 


m=  1  n=l 
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which  is  a  sum  of  all  the  lower  triangular  elements  of  A N+p.  By  making  a  change  of 
variables  according  to  (5.21),  these  lower  triangular  elements  can  also  be  summed  along 
diagonals.  That  is,  (5.28)  becomes 


M  m 


M 


M  M-j+l 

X  ^N+p' =  X  X  A N+p‘,m,n  EE  ^■N+p;j-l+k,k  ~  XDS(A  N+P)j. 

/?'=!  m=  1  n=  1  7=1  k=  1  7=1 


(5.29) 


Letting  p  =  P,  gives  the  majorization  condition  (5.24).  In  order  to  obtain  the  remaining 
condition  (5.25)  for  p  =  1, . . . ,  P  -  1,  we  first  show  that  DS(A N+p)j  =  0  whenever  j  >  p. 
To  do  this,  note  that  for  any  k  -  1, . . . ,  M  -  j  +  1,  by  Definition  33, 


A N+p:j-\+k,k  1  Pf>(/f\,+/):/- 1  +k  )  I A  —  ^([0,  ^-N+pJ-l+k]  ^  (P^k ,  &k—  1  ]  )  •  (5.30) 

Since  the  continued  outer  eigensteps  {{Tw+p;m}“=|  }^=|  must  interlace  (Definition  24.(iii)), 
we  have 

^■N+p;j—l+k  —  'f ,V+/)- 1 :/—  2— ‘  ’  —  T N’J—l—p+k  ~  & j—\—p+kt  (5.31) 

for  M  >  j-l-p+k>  1.  Continuing,  since  j  >  p,  we  have  j  -  1  >  p  and  so 
j-l-p+k>  p  -  p  +  k  =  k.  Thus,  (5.31)  becomes  An+p-j-  i+k  ^  ab  at  which  point 
(5.30)  becomes  zero,  and  so 

M-j+l 

DS(A N+p')]  —  ^  ^  Am+p-j-i+kk  —  0, 

k=  i 

whenever  j  >  p,  as  claimed.  Returning  to  (5.29),  the  fact  that  DS(A N+P)j  =  0  whenever 
j  >  P  gives 

p  M  mm{p,M } 

J]pn+P'  =J]DS(AN+P)j=  £  DS(A  N+p)j.  (5.32) 

P’= 1  7=1  7=1 

In  order  to  obtain  the  final  majorization  condition  (5.25),  we  also  claim  that 
DS(A;v+p);  <  DS(A  N+p+l)j  for  p  =  0,  ...,P  -  1  and  j  =  1  To  see  this,  we  use 

the  fact  that  {AN+p;m}%=  t  E  {AN+P+ i;m}^=v  specifically  that  ^N+p^m  —  ^N+p+\’,m  —  ^ ~N+p;m-l • 
Note  that  by  Definition  33, 


A N+p;m,n  ~  ^([0?  ^N+p^nA  D  (Xn—\\)  ^  ^([0,  ^N+p+\‘,m\  D  (Xn—\\)  —  A -N+p+\\m,n-  (5.33) 
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Coupling  (5.33)  with  Definition  37,  we  see  that 

M—j+l  M-j+l 

DS(A N+p)j  -  X  A-N+p-,j-l+k,k  ^  ^  ]  A-N+p+l;j-l+k,k  ~  DS(A N+p+l)j, 

k=  1  A=1 

as  claimed.  Indeed,  having  DS(A n+p)j  <  DS(A N+p+i)j,  (5.32)  becomes 

p  minjp.M}  mm{p.M)  min{p,M| 

YjVn+p'  =  J]  DS(A N+p)j  ^  DS(A N+p)j  —  DS(A)y, 

P'= 1  7=1  7=1  7=1 

which  is  the  final  majorization  condition  (5.25)  for  p  =  1, . . .  ,P  -  1.  □ 

5.3  Parametrizing  continued  eigensteps 

In  the  final  section  of  this  chapter,  we  characterize  sequences  {Am}ffI=l  which  are  ( a,p )- 
constructible.  As  discussed  in  the  beginning  of  the  previous  section,  solving  this  problem 
involves  finding  a  valid  sequence  of  continued  outer  eigensteps  for  given  nonnegative 
nonincreasing  sequences  {Am}^l=v  a  =  { am }^= , ,  and  p  =  \pN+/)}/i‘)=l .  Having  just  shown 
the  necessity  of  majorization  in  order  for  a  sequence  {A.m}‘ Jf=1  to  be  (a,yu)-constructible,  in 
this  section,  we  prove  the  converse  result,  namely  that  if  {DS(A);}^!  majorizes  {pN+p}pp=i, 
then  a  corresponding  sequence  of  continued  outer  eigensteps  {{AN+p-m}I^l=l}pp=0  exists. 

Recall  from  Chapter  4  the  simpler  case  when  no  initial  spectrum  is  given,  that  is, 
am  =  0  for  all  m.  There  we  showed  the  sufficiency  of  majorization  using  the  Top  Kill 
algorithm.  Top  Kill  removed  as  much  area  as  possible  from  the  top  levels  of  the  staircase 
(see  Figure  4.1)  subject  to  the  interlacing  and  trace  constraints.  In  a  similar  fashion  we 
develop  an  algorithm  that  will  show  that  majorization  is  sufficient  for  a  sequence  {Tm(“=| 
to  be  (a,/i)-constructible;  however,  this  time  we  remove  as  much  area  as  possible  from 
the  outermost  diagonals.  We  now  introduce  this  generalization  of  Top  Kill,  called  Chop 
Kill.  This  algorithm  gives  an  explicit  construction  of  a  feasible  sequence  of  continued  outer 
eigensteps  whenever  {DSIA);}^  majorizes  {pN+p}p=r 

Theorem  40.  Given  {Tw+p;m}^=|  e  adm(a)  and  for  which  !DS(Av+/,)/},/v^l  >  {pN+P'}pp,=l  the 
following  Chop  Kill  algorithm  constructs  { An+p- 1 Jn } ')f= ,  e  adm(or)  such  that  { /i,y+/)_  ]  -m}^= , 
E  {^N+p\m\m= 1  and  {DS(Aw+p-l)m}^f=1  >  {pN+pfp’lf 
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01 

het  6\:^p_\mjl  Q!n—  1 

an  for  n  >  m 

02 

Vm,\  '■=  Pn+p 

03 

for  j  = 

04 

for  k  =  M  -  j  +  1 , 

...,1 

05 

6  -  min{v7_i+fe^,  Aw+p;;_i+^ 

06 

^N+p-l,j-l+k,k 

:=  A  N+p;j-l+k,k 

07 

ifk  >  1 

08 

Vj-2+kJc-l 

'■=  vj-l+k,k  -  <5 

09 

else 

10 

VM.M-j+2 

:=  Vj-l+kjc  ~  <5 

11 

end 

12 

end 

13 

end 

14 

Furthermore,  given  any  nonnegative  nonincreasing  sequences  {am\^=v  {Am}^=1  and 
{FN+p}p=  i  such  that  {DS(A )j}f=l  >  {jxN+p}pp=v  define  AN+P;m  :=  Am  for  every  m  =  1 
and  for  each  p  =  P, . . .  ,2,  consecutively  define  {AN+p-i:m}^=l  according  to  Chop  Kill.  Then 
l!A\,+/>:m},)f=|  }//,'=0  are  continued  outer  eigensteps. 


In  order  to  prove  Theorem  40,  we  will  need  the  following  lemma: 


Lemma  41.  Given  {AN+p;m}ffl=l  e  adm(rr)  and  for  which  {DS(AN+P)  j}¥=1  >  {pN+P>\Pp,=v  if 
|  /i,v+/)_  i  ;m  }^= !  6  adm(a)  is  constructed  according  to  the  Chop  Kill  algorithm  of  Theorem  40, 
then 


A 


<  A 


N+p;m+l,n  —  *^N+p-l;m,n  —  **~N+p;m,ni 


<  aa 


(5.34) 
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for  all  m  =  1  and  n  =  1 .....  M  +  1.  Moreover,  there  exists  an  index  (jo,  k0)  e  Jf 

such  that 


A 


N+p-l-,j-l+k,k 


I  ^-N+p-,j+k,k 


if(j,k )  >  (Jo,  k0). 


(5.35) 


^N+p-vj-i+k,k  if  (j,  k)  <  (jo,  ko), 

where  “(j,  k)  >  (jo,  ko)  ”  means  either  j  >  jo  or  if  j  =  jo  then  k  >  k0.  Moreover,  in  the  case 
where  (j,  k )  =  (jo,  ko): 


M-j0+l 


A 


N+p-l:j0-l+k0,k0 


~dN+p  +  DS(A^+P)y0+1  +  I  A 


N+p;jo~l+k,k 


k=k0 


M-jo+1 

-  E 

k=ko+l 


A 


N+p-,j0+k,k- 


(5.36) 


Before  proving  Theorem  40  and  Lemma  41,  we  give  an  example  in  order  to 
demonstrate  how  the  Chop  Kill  algorithm  works. 


Example  42.  We  use  the  Chop  Kill  Algorithm  of  Theorem  40  in  order  to  construct  a 
sequence  of  continued  outer  eigensteps  for  a  frame  obtained  by  adding  P  -  4  additional 
vectors  to  a  set  of  preexisting  vectors  in  M3.  For  this  example,  our  initial  starting  sequence 
a  -  {a'm|3i=|  is  given  by  the  final  sequence  of  eigenvalues  in  Example  15  of  Chapter  4. 
Specifically,  we  let  {ai,a2,aj  =  {|,  |,|}.  The  desired  final  spectrum  is  {Ti,  X2,  T3}  = 
{■j,  b  1}  with  the  additional  vector  lengths  given  by  {iuN+l,iuN+2,iiN+3,nN+4}  =  {2, 1,  ±,  ±}. 
Our  goal  is  to  complete  the  following  table  according  to  the  rules  of  continued  outer 
eigensteps  (Definition  24): 


N  +  p  0  1 

An-.  3  l  ? 

|  ? 

An-,1  l  ? 

To  be  precise,  we  must  pick  {/hv+i;m}3=1, 
interlacing  conditions, 


2  3  4 

?  ?  1 

?  ?  2 

4 

?  ?  12 

4 

{/hv+2;»i}m=i  and  {/hv+3;m}m=i 


(5.37) 


that  satisfy  the 


}  —  —  {/^N+2,m}m=\  E  {/^N+3,m}m=\  E  {  4  ?  4?  1}> 
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as  well  as  the  trace  conditions, 


djV+l;l  +  /hv+i;2  +  dw+i;3  -  ^  am  +  fJ.N+1  -  5, 

m=  1 

3  2 

/lyv+2;l  +  An+2;2  +  A n+2-,3  =  ^  ^  ^/V+p'  =  6, 


m—  1  /?'  =  ! 

3  3 

dw+3;l  +  dW+3;2  +  d^+3;3  =  ^  am  +  ^ 

m=  1  p'= 1 

3  4 

dw+4;l  +  /bv+4;2  +  d/v+4;3  =  ^  a„,  + 

m=l  //=! 


,,  -25 

PN+p'  ~  4  j 


//  -  13 

H-N+p'  9  • 


The  trace  condition  means  the  sum  of  the  values  in  the  N  +  p  column  is  £m=i  a,,,  + 
X|/=1  Pn+p>,  while  the  interlacing  condition  means  that  any  value  AN+p-m  is  at  least  the 
neighbor  to  the  upper  right  AN+P+  i;m+j  and  no  more  than  its  neighbor  to  the  right,  AN+P+  i;m. 

We  can  view  the  task  of  completing  (5.37)  as  iteratively  building  a  staircase.  Our 
goal  is  to  build  on  top  of  an  existing  three-step  staircase  with  steps  of  length 
(Figure  5.1(a))  so  that  the  resulting  bottom  level  has  length  the  second  level  has  length 
|,  and  the  top  level  has  length  1  (Figure  5.1(b));  the  initial  and  final  profile  of  each  staircase 
is  outlined  in  black  in  each  of  the  sub  figures  of  Figure  5.1.  Similar  to  Top  Kill  in  Chapter  4, 
Chop  Kill  works  backwards  chipping  away  at  the  three-level  staircase  in  Figure  5.1(b)  until 
all  of  the  light  gray  area  is  removed.  To  determine  which  areas  should  be  removed  first, 
we  chop  up  the  staircase  into  blocks  according  to  the  initial  spectrum  a  (indicated  by  the 
dotted  lines  in  Figure  5.1(c)),  and  then  place  labels  on  each  gray  block  corresponding  to 
its  position  relative  to  the  initial  spectrum  a.  Blocks  with  label  “1”  in  Figure  5.1(d)  are 
one  step  above  the  initial  spectrum  a  while  blocks  with  label  “2”  are  two  steps  above  the 
initial  spectrum.  As  you  can  see  in  Figure  5.1(d),  areas  with  the  same  label  form  a  diagonal 
along  the  profile  of  the  staircase.  Chop  Kill  works  by  removing  diagonal  “2”  first  followed 
by  the  diagonal  labeled  “1”.  Chop  Kill  derives  its  name  from  this  process  of  chopping  up 
the  staircase  according  to  a  and  then  “killing”  off  as  much  as  possible  from  the  outermost 
diagonals. 
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(c)  (c) 

Figure  5.1:  Given  existing  spectrum  a  =  {ai,  a2,  (*3}  =  {\,  7}  (a),  we  add  4  additional 

vectors  whose  lengths  are  given  by  {//Ar+t,/W+2»/W+3>Ahv+4}  =  {2, 1,|,|}  so  that  the 
completed  frame  has  spectrum  {di,  A2,  /I3}  =  {-j,  |,  1}  (b).  Our  goal  is  to  build  a  sequence  of 
continued  outer  eigensteps  for  the  gray  area  in  (b).  To  determine  the  priority  in  which  area 
should  be  removed,  we  chop  up  the  staircase  into  blocks  according  to  the  initial  spectrum  a 
(indicated  by  the  dotted  lines  in  (c)),  and  then  place  labels  on  each  gray  block  corresponding 
to  its  position  relative  to  the  initial  spectrum  a  (d).  Blocks  with  label  “1”  in  (d)  are  one 
step  above  the  initial  spectrum  a  while  blocks  with  label  “2”  are  two  steps  above  the  initial 
spectrum. 


We  now  attempt  to  build  this  staircase,  working  backwards  until  all  of  the  gray  area  has 
been  removed.  As  just  mentioned,  our  strategy  for  removal  is  to  remove  diagonal  “2”  first, 
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0  1  i  O  -|  iJ  11  o  13 

4241424Z424'^4 

(a) 


n  1  1  3  1  537995Uo13 

U4241424^424'34 

(b) 


5  3  7 


n  1  1  3  1  537995Uol3 

^424^424^424^4 

(c) 


n  1  1  3  1  537995Uol3 
^424^424^424^4 

(d) 


Figure  5.2:  Iteratively  building  a  sequence  of  continued  outer  eigensteps  for  {/lj,  A2,  d3}  = 
{-j,!’1)  where  a  =  {aua2,a3}  =  {\,\,\}  and  {fiN+i,fiN+2,fiN+3,^N+4}  =  [2,1, 
Beginning  with  the  final  desired  staircase,  we  work  backwards  to  the  initial  staircase. 
That  is,  we  chip  away  at  the  three-level  staircase  (a)  to  produce  (b),  chip  away  at  (b)  to 
produce  (c),  and  then  finally  chip  away  from  (c)  to  produce  (d).  In  each  step,  we  do  this 
by  removing  as  much  as  possible  from  the  top  diagonals  before  turning  our  attention  to  the 
lower  diagonals,  subject  to  the  interlacing  and  trace  constraints.  We  refer  to  this  algorithm 
for  iteratively  producing  {AN+P-  i;m}^f=1  from  {AN+p-m}^=l  as  Chop  Kill. 


followed  by  diagonal  “1”.  We  begin  with  the  final  desired  spectrum  {/bv+4;i,  AN+4:2,  /fv+4;3 }  = 
{■j,  1}  (Figure  5.2(a)).  Observe  in  Figure  5.2(a),  that  the  highest  portion  of  diagonal  “2” 

is  at  the  third  level  of  the  staircase.  As  such,  we  remove  iuN+4  =  \  units  of  area  from 
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this  level  resulting  in  a  staircase — the  darker  shade  in  Figure  5.2(b) — which  has  levels  of 
lengths  {/hv+3;i,/hv+3;2,  /hv+3;3}  =  {-j’  In  the  next  step  we  remove  a  block  of  area 

Pn+3  =  \  from  the  second  level  of  the  staircase.  We  remove  from  the  second  level  since 
it  contains  the  highest  instance  of  diagonal  “2”  that  has  yet  to  be  removed.  This  results 
in  a  staircase  with  lengths  {An+2-\,  A.n+i-,2,  A-n+i-,3)  =  {-j,  2,  |}-again,  the  darker  shade  in 
Figure  5.2(c).  In  the  third  step,  we  apply  this  same  philosophy,  this  time  removing  two 
blocks  of  area  from  the  second  and  third  levels  whose  total  area  will  be  pN+2  =  1.  Since 
there  is  still  part  of  “2”  left  at  the  second  level,  we  chip  away  this  remaining  I  units 
of  area  first.  At  this  point,  all  of  the  “2”  area  has  been  removed,  which  leaves  |  units 
of  area  to  remove  from  diagonal  “1”.  Accordingly,  we  chip  away  |  units  of  area  from 
level  three  and  an  additional  |  units  of  area  from  level  two  resulting  in  a  staircase  with 
levels  =  {-j,  |,  1}  (Figure  5.2(d)).  In  the  final  step,  we  remove  the 

remaining  areas  of  diagonal  “1”  totaling  /iN+\  =  2  units  of  area.  This  process  produce  the 
following  valid  sequence  of  continued  outer  eigensteps: 

N+p  01234 


113  3 

2  2  4  4 


(5.38) 


3  5 

4  4 


o  2  2 

Z  4  4 


7  13  13  13  13 

4  4  4  4  4 


In  the  analysis  that  follows,  we  show  that  the  Chop  Kill  algorithm  does  indeed  produce  this 
same  sequence  of  continued  outer  eigensteps. 

Now  that  we  know  intuitively  how  the  Chop  Kill  algorithm  works,  we  repeat  the 
process  of  constructing  (5.38)  using  the  algorithm  given  in  Theorem  40.  We  will  apply 
Chop  Kill  three  times,  once  for  each  of  the  undefined  columns  in  (5.37).  As  with  Top  Kill, 
we  work  backwards,  producing  the  third  column  from  the  fourth,  then  the  second  from  the 
third,  and  so  on.  For  each  column,  the  Chop  Kill  algorithm  first  finds  the  spectral  partition 
matrix  for  the  eigenvalues  belonging  in  that  particular  column  (Lines  01-13).  For  this 
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example,  M  =  3,  so  each  spectral  partition  matrix  that  is  constructed  will  be  of  size  3x4. 
Chop  Kill  defines  the  entries  of  each  spectral  partition  matrix  one  at  a  time — sweeping 
through  each  lower  diagonal  from  right  to  left.  As  a  final  step  (Line  14),  Chop  Kill  sums 
across  the  rows  of  the  spectral  partition  matrix  produced  by  Lines  01-13  to  obtain  the 
desired  eigenvalues. 

We  begin  with  p  =  4,  and  use  the  Chop  Kill  algorithm  to  construct  A;V+3,  the  spectral 
partition  matrix  for  {/fv+ 3-m}^n=l  ■  In  order  to  do  so,  we  need  to  know  A;V+4,  since  the  value  of 
6  (Line  05)  depends  in  part  on  a  differences  between  matrix  entries  of  AN+4.  For  the  values 
of  {4,„(^=l  chosen  in  this  example,  we  have  already  shown  in  Example  34  how  to  construct 
this  spectral  partition  matrix  using  the  function  P„  (Definition  33): 

3  i  1  I 

2  1  4  2 

1  1  I  I 

2  4  2 

o  I  I  I 

4  4  2 

At  Line  01,  we  begin  by  setting  Aw+3;m;„  =  an-\  -  an  for  n  >  m.  In  essence,  this 
is  done  to  ensure  that  the  resulting  matrix  Aw+3 — which  much  be  a-admissible — satisfies 
Definition  35.(iii).  This  completes  the  upper  triangular  portion  of  A/V+3  as  follows: 

?  1  i  i 

4  2 

9  9  I  I 

‘  4  2 

9  9  9  I 

2  J 

Next  at  Line  02,  we  set  v3j  =  pN+4  =  Since  we  are  removing  a  total  area  of  pN+ 4, 
these  “v”  quantities  track  how  much  area  is  left  to  be  removed  at  future  iterations  of  the 
“for-loops”  (Lines  03-13).  The  for- loops  set  the  values  of  the  diagonals  of  AiV+3  working 
from  the  outermost  diagonal  to  the  main  diagonal,  and  then  from  right  to  left  within  each 
diagonal.  Referring  to  the  matrix  indexing  given  in  (5.22),  the  Chop  Kill  algorithm  sets  the 
values  of  AN+3  in  the  following  order: 

(3,1),  (3,2),  (2,1),  (3,3),  (2,2),  (1,1). 
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Beginning  with  j  -  3  (Line  03),  the  outermost  diagonal  only  consists  of  one  element  and 
so  k  =  1.  At  Line  05,  we  set 

6  =  min{v31,  Aat+4-3 j  -  Av+4;4,i}  =  min{±,0}  =  0, 

where  Aw+4;4,:  :=  0,  which  gives  A/v+3;3,i  =  A at+4;3,i  -5=0-0=0  (Line  06).  The 
next  value  of  v  is  defined  by  Lines  07-11  depending  on  the  value  of  k,  and  since  k  =  1, 
V3,2  '■=  v3, i  -  8  =  |  -  0  =  At  the  end  of  the  for-loop  for  j  =  3,  we  have: 

?  1  i  i 

4  2 

?  ?  i  i 

•  4  2 

o  ?  ?  i 

Next  for  j  =  2,  the  Chop  Kill  algorithm  sweeps  from  right  to  left  along  the  2nd  lower 
diagonal  of  A w+3.  The  2nd  diagonal  consists  of  two  elements  and  so  k  ranges  from  1  to  2. 
Sweeping  from  right  to  left  along  this  diagonal  is  equivalent  to  working  backwards  from 
k  =  2, . . . ,  1.  When  k  =  2,  Line  05  gives 

8  =  min{v3;2,  Aw+4;3;2  -  An+4.A2}  =  min{|,0}  =  J, 

which  means  that  |  units  of  area  will  be  removed  from  Aw+4;3-2,  i.e.,  Aa?+3;3i2  =  A;v+4;3-2-d  = 
|  |  =  0.  Since  \  unit  of  area  is  removed  and  jjN+A  =  we  expect  the  next  value  of  v  to 

be  zero  since  all  of  the  area  has  already  been  removed;  this  indeed  is  true: 

v2,  i  =  v3,2  -  £=  |-  |=0. 

In  fact,  once  the  value  of  v  becomes  zero,  all  remaining  values  of  v  computed  in  subsequent 
iterations,  particularly  v3  3,  v2>2,  and  vu,  will  be  zero  as  well.  As  such,  8  (Line  05)  will  also 
become  zero.  This  simplifies  the  calculations  for  the  remainder  of  the  iterations  and  Line 
06  will  simplify  to  AN+P-\,j-\+k,k  '•=  ^-N+P-j-i+k,k-  Notice  in  all  the  calculations  up  to  this 
point,  8  =  AN+p;j_]+kyk  -  AN+p.j+kyk,  giving  that  AN+p_Uhl+kyk  :=  AN+p.j+kyk.  Using  this  fact, 
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we  expect  AiV+3  to  be 

3  t  1  t 

2  1 2  4  2 

5  1  i  5  (539) 

0  0  J  I. 

Later  we  will  show  that  this  “transition”  point  (j0,  k0)  =  (3, 2)  e  27  is  unique  and  that 

all  the  matrix  entries  of  Aw+P_i  can  be  defined  according  to  this  unique  index  pair.  We 
formalize  this  result  in  Lemma  41  and  use  it  to  prove  Theorem  40.  While  knowing  this 
transition  index  greatly  simplifies  our  calculations,  its  value  cannot  be  determined  explicitly 
in  advance,  and  so,  the  Chop  Kill  algorithm  is  still  the  main  tool  we  use  in  order  to  construct 
a  sequence  of  continued  outer  eigensteps. 

To  verify  that  (5.39)  is  indeed  true,  we  return  to  our  example  and  pick  back  up  with 
the  case  when  j  -  2  and  k  =  1 .  In  this  case, 

8  =  min{v2,i,  A/v+4-2,1  -  Aw+4;3j1}  =  min{0,  \  -  0}  =  0, 

which  is  what  we  expected  since  all  of  / un+4  was  removed  in  the  previous  iteration.  Line  06 
gives  that  An+3-,2,\  ■=  A n+a-x\  =  ^  and  at  Line  08,  we  have  v3;3  =  0,  which  finishes  the  2nd 
diagonal  of  AN+3: 

?  1  I  i 

4  2 

1  ?  i  I 

2  •  4  2 

0  0  ?  | 

What  remains  is  the  main  diagonal  which  corresponds  to  the  case  when  j  =  1. 
Since  this  diagonal  has  three  elements,  the  Chop  Kill  algorithm  works  backwards  from 
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k  =  3, . . . ,  1.  We  summarize  these  calculations  in  the  following  table: 


j  k  6  A  w+3  j_  | +/,^  V  Ajv+3 

?  1  i  I 

4  2 

1  ?  I  I 

2  •  4  2 

ooli 

?  1  I  I 

4  2  (5.40) 

;  i  n 
0  0  1  l 

3  1  1  1 

2  1  4  2 

|  1  \  \ 

0  0  1  l 

In  each  of  these  last  few  steps,  the  new  value  in  the  A,v+3  matrix  is  set  to  the  value  in  the 
corresponding  location  of  A/V+4.  As  we  expected,  all  values  of  6  and  v  in  columns  three  and 
five  of  (5.40)  are  set  to  zero  since  all  of  pN+4  has  already  been  removed.  At  Line  13  of  the 
Chop  Kill  algorithm,  we  have  indeed  produced  (5.39),  the  spectral  partition  matrix  Aw+3. 
Having  this  spectral  partition  matrix,  we  now  sum  across  its  rows  (Line  14),  which  gives 
the  fourth  column  of  (5.38). 

Next  we  repeat  the  Chop  Kill  algorithm  for  p  =  3  and  construct  AN+2  by  removing  a 
total  of  pN+?,  =  |  units  of  area.  Again  we  work  from  the  outermost  diagonal  and  from  right 
to  left  along  each  diagonal.  For  the  outermost  diagonal,  j  =  3,  the  Chop  Kill  algorithm 
produces: 

j  k  6  AN+2j-\+k,k  y  An+2 

?  1  I  i 

4  2 

3  10  An+2,3,1  :=  0  V3,2  :=  \  1  1  ^  \ 

0  ?  ?  \ 

For  the  next  iteration  j  =  2,  the  Chop  Kill  algorithm  defines  the  second  diagonal  as  follows: 


13  0  Aat+3^3  :-  |  v2,2  :-  0 


120  aw+3,2,2  i  y\, i o 


110  Aat+3  !  !  :-  |  v3  4  :-  0 
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k  8  A 


2  1 


N+2,j-l+k,k 


A 


N+2 


2  2  0  Ajv+2,3,2  0  V2 1  t 


Aat+2,2,1  2  V33  0 


?  1  I  I 

4  2 

?  ?  I  I 

•  4  2 

°  °  ?  | 


?  1 


1  1 
4  2 


1  9  1  1 
4  •  4  2 


0  0  ?  \ 


Notice  in  these  calculations  when  j  =  2  and  k  =  1,  |  units  of  area  are  removed  from 
the  (2, 1)  entry  of  Aw+3.  Since  all  of  /fv+3  is  removed  at  this  iteration,  the  next  value  of  v. 


V33,  is  set  to  zero.  In  this  case,  the  unique  index  (jo,  ko)  which  corresponds  to  when  all  of 
/r,v+3  has  been  removed  is  (2, 1).  Finally,  for  the  last  iteration  when  /  =  1 : 


This  yields  the  resulting  matrix  for  AN+2 : 


An+2 


1  1 
4  2 

1  1 
4  2 


0  0  I 


1 

2 
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Summing  across  the  rows  of  AN+2  (Line  14)  yields  the  third  column  of  (5.38)  as  desired. 

Finally,  we  repeat  this  process  for  p  =  2,  removing  pN+2  =  1  in  order  to  construct 
Aw+n  The  evolution  of  A,v+|  at  each  iteration  of  the  Chop  Kill  algorithm  is  summarized  in 
the  following  table: 


Summing  across  the  rows  of  the  resulting  matrix  A;V+i  does  in  fact  produce  the 
eigensteps  in  the  second  column  of  (5.38). 
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For  p  =  1,  the  Chop  Kill  algorithm  produces  A,.v  which  is  already  given  by  the  initial 
sequence  of  eigenvalues  a.  It  is  straightforward  to  verify  that  the  Chop  Kill  algorithm 
produces  the  same  AN  that  can  found  by  applying  P„.  given  by  (5.13),  to  a.  While  these 
calculations  seem  very  tedious,  it  is  very  easy  automated  using  MATLAB.  The  code  can  be 
found  in  the  appendix. 

Note  that  one  application  of  Chop  Kill  generates  {/fv+p_1;m)^=]  from  {/1a r+p-m}^=l.  Just 
as  demonstrated  in  the  example,  one  may  start  with  any  {AN+p-m}^l={  :=  {/lm}“=1  and  apply 
the  Chop  Kill  algorithm  P—  1  times  in  order  to  produce  a  valid  sequence  of  continued  outer 
eigensteps  \{^N+P-m\m=\  lp=0  that  satisfies  Definition  24. 

Having  discussed  how  Chop  Kill  works,  we  now  prove  Lemma  41  first  followed  by 
the  proof  of  Theorem  40. 

Proof  of  Lemma  41.  For  the  sake  of  notational  simplicity,  we  denote  B  :=  A  /V+/,_  |  and 
C  :=  AN+p  meaning  Bjk  :=  AN+p_\.pk  and  Cjhk  :=  AN+/KjM.  We  begin  by  proving  (5.34). 
Specifically  we  wish  to  show  that 


Cm+l.n  —  Pm.n  —  Cmn. 


(5.41) 


To  do  so,  we  consider  the  case  when  n  >  m  and  the  case  when  n  <  m.  In  the  case  that 
n  >  m,  the  fact  that  C  6  Adm(a)  implies 


C, 


m+l,n 


<cn 


&n—  1 


(5.42) 


From  Line  01  of  the  Chop  Kill  algorithm,  Bmn  =  ,  -  an;  substituting  this  into  (5.42) 

gives 

Cn+i,/i  —  Cmn  —  (yn~  i  an  —  Bmn  —  (r  n-  \  Qn  —  Cmn, 

and  so  (5.41)  holds  in  the  case  that  n  >  m.  On  the  other  hand  if  n  <  m,  note  that  Line  05 
gives  6  <  Cj- i+ktk  -  Cj+k'k.  Substituting  this  for  8  at  Line  06  gives 

Bj-l+k,k  —  Cj-l+kk  {C j—\+k-k  Cj+k;k)  ~  Lj+k:k •  (5.43) 
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We  also  have  that  8  <  v;-_i+^  which  follows  from  Line  05  as  well.  This  implies  that  all 
new  values  of  v  defined  at  Lines  08  and  10  are  nonnegative;  specifically,  Vj-2+k,k  >  0  and 
vM,M-j+ 2  ^  0  for  all  j  <  k.  Combining  this  with  the  fact  that  Cmjl  -  C,„_ i+,un  >  0 — due  to 
the  fact  that  C  e  Adm(a) — implies  8  >  0  (Line  05).  As  such.  Line  06  gives 

Bj- 1  +k,k  =  C/- 1  +k,k  -  8  <  Cj-i+k-k-  (5.44) 

Combining  (5.43)  and  (5.44)  and  making  a  change  of  variables  according  to  (5.20),  we 
have  that  (5.41)  holds  in  the  case  that  n  <  m. 

Next  we  prove  (5.35).  First,  we  place  an  ordering  on  the  set  of  all  pairs  of  indices  J~. 
Specifically  we  write  (j,  k )  >  (/,  k')  if  either  j  >  f  or  if  j  -  f  and  k  >  k' .  Notice  that  the 
algorithm  computes  the  entries  of  B  from  those  of  C  in  this  order  working  from  greatest  to 
least.  That  is,  at  first  it  defines  values  of  B  at  ( M ,  1),  then  at  ( M ,  2),  ( M -  1, 1),  etc.,  finishing 
at  (1, 1).  We  now  let  (j0,  k{))  be  the  maximal  pair  (j,  k )  such  that 

V/- 1  i-k.k  —  Cj-k+ !  .k  Cj-kk  •  (5.45) 

To  show  that  (jo,k0)  is  well-defined,  we  must  show  that  (5.45)  holds  for  at  least  one 
(j,  k)  6  J~.  Suppose  to  the  contrary  that  there  is  no  such  pair  (  /,  k)  such  that  (5.45)  holds, 
That  is, 

V j—\+k,k  ^  C/.  |  +kM  Cj+k,k-  (5.46) 

In  particular  for  (j,  k)  =  (1, 1) 

yu  >  Ci,i  _  C24.  (5.47) 

Moreover,  for  (  j,  k)  >  (1, 1),  (5.46)  implies  that  8  of  Line  05  is  given  by 

8  =  C )-i+k,k  ~  C j+k,k- 

As  such,  progressing  through  all  but  the  last  step  of  the  Chop  Kill  algorithm,  namely 
progressing  from  (  j,  k )  =  (M,  1)  down  to  (  j,  k )  =  (1, 2),  gives 

M  M—j+ 1  M 

V1.1  :=  Bn+p  ~  ^  ^  (Cj-i+k,k  ~  Cj+k,k)  ~  Ck,k  —  Ck+i,k)-  (5.48) 

j= 2  k=  1  k=2 


117 


Combining  (5.47)  with  (5.48)  and  rearranging  gives: 

M  M—j+ 1 

l^N+p  ^  ID  (C  j- 1  +&,£  C  j+k,k)  •  (5.49) 

7=1  *=1 

Next  we  make  a  change  of  variables  according  to  (5.20).  This  allows  us  to  rewrite  (5.49) 
as  follows: 


M  m 


M  m 


M+ 1  m- 1 


I^N+p  ^  ^  ^  ^  ^(C/it,n  Cm+l,n)  ~  ZZ  Cm,n  -ZZC- 

m=  1  n— 1 


ra=l  ti=1 


m=2  n=l 


Since  CM+i,„  :=  0,  (5.50)  further  reduces  to 


Mm  M  m- 1 

«*,>££c«-££c- 

772=1  71=1  711=2  71=1 

Mm  M  Tii-l 

—  ^*i,i  ^  ^  ^  ^  Cm  n  ^  ^  ^  ^ 

711=2  71= 

M 

=  Ci,i  +  ^  cn 


711=2  71=1  711=2  71=1 

M 


m=2 


(5.50) 


DS(C)l 


This  contradicts  our  assumption  that  {DS(C);}^1  >  {/iW+P}p=1 — specifically  this  contradicts 
the  fact  that  DS(C)i  >  //v+ 1  >  A/,v+/)-  As  such,  (5.45)  holds  for  at  least  one  pair  (j,  k )  and  so 
(jo,  ko)  is  well-defined. 

We  now  prove  (5.35),  that  is,  for  all  but  the  (jo,  k0) th  entry,  the  “new”  spectral  partition 
matrix  entry  is  either  a  copy  of  the  existing  entry  in  that  spot  or  a  copy  of  the  entry  directly 
below  it.  To  see  this  fact,  note  that  if  (/,  k)  >  (jo,  ko),  then  since  (jo,  ko)  is  the  maximal  pair 
such  that  (5.46)  holds,  then  (5.46)  does  not  hold  for  our  particular  choice  of  (j,  k),  implying 
6  at  Line  05  is  6  =  Cr ,  +kk  -  CJ+kk.  At  this  point,  Line  06  gives 


B, 


j- 1  +k,k 


=  c 


j-l+k,k 


0 c 


j-i+k,k 


C j+k,k)  -  C 


j+k,k. 


namely  (5.35)  in  the  case  that  (j,  k )  >  (jo,  ko).  To  prove  (5.35)  holds  in  the  remaining  case 
where  (j,  k)  <  (jo,  ko),  note  that  at  the  (jo,k0) th  step  of  the  Chop  Kill  algorithm,  the  fact 
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that  (5.45)  holds  implies  6  =  v7-_i+^.  Thus  subsequent  values  of  v  defined  at  Lines  08  and 
10,  namely  Vj-2+k,k  when  k  >  1  and  vm.m-j+i  when  k  -  1,  will  be  zero.  As  such,  for  all 
remaining  steps  of  the  Chop  Kill  algorithm  (namely  those  where  (j,  k )  <  (jo,  ko)),  Line  05 
gives  6  =  0  at  which  point  Line  06  gives  (5.35)  in  the  case  that  (j,  k )  <  (jo,  ko). 

Finally,  we  must  show  that  (5.36)  holds  in  the  case  that  (j,  k )  =  (jo,  ko).  Note  that  since 
(  jo,  ko)  is  the  maximal  index  such  that  (5.45)  holds,  at  Line  06  of  the  Chop  Kill  algorithm, 

BjQ-i+k0,kQ  =  Cj0-i+ko,ko  —  6  =  Cj0-i+k0,ko  —  VjQ-l+k0,ko  (5.51) 

where  vjo-1+koM  is  given  by 

M  M-j+ 1  M-jo+l 

Vjo~l+ko,ko  ~  H-N+p  E  E  (C j-\+k,k  C j+k,k)  ^  ^  fC/(i_ |  +k.k  t'jn-k.k ) ■  (5.52) 

7=70+1  k=  1  k=k0+ 1 

Substituting  (5.52)  into  (5.51)  gives  (5.36)  as  claimed.  □ 

Proof  of  Theorem  40.  For  the  sake  of  notational  simplicity,  we  denote  {/?,„  ( ^=]  :  = 

{ /fv— /j- i :m \ tn  p  Bjjc  •  AA:+/)_ j -  jk,  B  .  A;y+a~ i ,  and  denote  •  ^N+p;m}m= 

Cjtk  :=  A N+P:j,k  and  C  :=  A,v+/).  First  we  show  that  B  6  Adm(a)  by  showing  that  B 
satisfies  all  three  properties  of  Definition  35.  Property  (iii)  follows  immediately  from  Line 
01  that  Bm  n  =  a„_i  -  an  for  n  >  m.  Property  (ii)  is  satisfied  as  a  result  of  Lemma  41 — in 
particular  from  (5.34)  .  Combining  (5.34)  at  m  and  (5.34)  at  m  -  1, 


Cm+l  ,n  —  Bn 


<  CmM  <  B 


m-l,n 


<  c 

—  ^  tu¬ 


rn—  1  ,n  • 


Comparing  the  second  and  fourth  terms  gives  property  (ii)  that  Bmn  <  Bm_]  n.  In  order 
to  show  property  (i)  holds  we  consider  several  different  cases.  To  be  clear,  assume  that 
Bm  ni  >  0  for  some  (mi,  nf).  We  must  show  that 

Bmun  —  (%n—l  (5.53) 

for  n  >  n\.  From  Line  01  of  the  Chop  Kill  algorithm,  (5.53)  immediately  holds  for  n  >  m\ 
and  so  it  suffices  to  prove  (5.53)  for  <  n  <  ni\.  In  particular  since  n\  <  m\,  the  index  pair 
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(mi,  ni)  6  M,  the  lower  triangular  part  of  B.  By  applying  the  coordinate  transform  (5.20), 
(mi, ni)  can  be  written  as: 


(m\,ni)  =  cr(ji,ki)  =  (jx  -  1  +ki,kx) 


for  some  unique  (j\,ki)  e  jj.  We  now  prove  (5.53)  by  separately  considering  the  case 
where  (ji,kx)  <  ( j0,k0 )  and  the  case  where  (juh)  >  (j0,k0).  If  (Juh)  <  C/o>*b)  then  by 
Lemma  41,  i+Jt)fc  =  Cy-i+^/t  for  all  ( /', /c)  <  (j\,k\).  As  such 


B 


m\,n 


=  c 


(5.54) 


for  all  77  >  77 1 .  Moreover  since  (  j\  ,k\)  <  (jo,k0),  then  the  assumption  that  Bjl_]+klj<l  - 
Bmuni  >  0  implies  Cmi  Ml  >  Bmuni  >  0  as  a  result  of  Lemma  41.  Combining  this  with  the 
fact  that  C  6  Adm(o')  we  have 


C, 


m\,n\ 


&n- 1 


(5.55) 


for  all  77  >  ni.  Putting  (5.54)  and  (5.55)  together  gives  =  ar„_i  -  an  for  all  n  >  nx  as 
claimed.  In  the  case  where  (ji,k\)  >  (jo,  ko),  Lemma  41  gives  Bmuni  =  Cmi+ 1>ni.  Note  that 
since  Bmuni  >  0,  this  immediately  rules  out  mx  -  M.  Since  Cmi+1,„,  =  Bmuni  >  0  where 
C  e  Admit*),  then 

C>iii  +  l,n  —  (%n—  1  ®n. 


for  all  77  >  77] .  By  Lemma  41 


U'n-l  O',,  —  Cmi+l,n  —  ^mi,n  —  Cmi,n  —  (%n—  1 

where  the  last  inequality  follows  from  C  6  Adm(cr),  in  particular  the  fact  that  the  rows  of 
C  are  cr-partitions  (see  Definition  28).  Thus  Bmun  =  an-i  -  an  for  all  n  >  nx  as  claimed  and 
so  B  6  Adm(cr). 
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Next  we  show  that  {(3 m}^=|  E  {y,„}^=| .  Using  the  results  of  Lemma  41  and  the  fact  that 
B  and  C  are  both  a-admissible,  we  have 

M+ 1  M+l 


7m+l  —  ^  ^  *  1 .//  —  ^  ^  Bm  n  [3  !r.  - 

m=  1  m=  1 

M+l  M+l 


Pm  ^  ^  Bm,n  —  ^  ^ 


y  m- 


(5.56) 

(5.57) 


m=l  m=l 

Putting  (5.56)  and  (5.57)  together,  ym+,  <  /3m  <  ym  and  so  {/3m("=|  E  {ym}"=1  as  desired. 

For  the  final  claim  that  {DS(fi)y}jf;1  majorizes  {//„}'^(  | ,  first  we  write  {DS(B)j}^=l  in 
terms  of  {DS(C)7}^1.  To  do  this,  we  use  the  result  of  Lemma  41  that  there  exists  an  index 
(jo,  k0)  6  3  such  that  (5.35)  and  (5.36)  hold.  In  the  case  that  j  <  jo, 

M-j+l  M-j+l 


DS (B)j  -  ^  Bj~\+k,k-  Z  Cj-\+k,k  -  DS(C)j. 


(5.58) 


k= i 


k= i 


On  the  other  hand,  in  the  case  that  j  >  jo. 


M-j+l  M-j+l 

ds (B)j  =  Z  Bj-i+kfi  =  z  Cj+k,k 

k=  i  k=i 

M-U+D-l 

Z  C(j+iyi+k,k  -  DS(C)y-+i. 


-  C M+l, M-j+l  + 


(5.59) 


k=l 


since  CM+ i,n  '■=  0.  Finally  in  the  case  where  j  =  jo,  DS (B)jo  becomes: 

M-jo-l  A'o-1  M-jo+l 

Z;  Bj0 ~\+k,k = Cj0-i+k,k + Bj0- i+k0,k0  +  Cj0+k,k- 

k=  1  k= 1  k=ko+l 

Substituting  (5.36)  into  (5.60), 

M  M  M-j0+l 


(5.60) 


DS0B);o  =  Yj  DS^J  ~  Z  DS(ClM  +  Z  C 


jo~l+k,k  BN+p 


7=7o+l 

M 


7=7 o+l 
M 


=  £  DS(C)j  -  £  DS(C)y+,  ■ 

7=70  7=70+1 

=  DS(C);o  +  DS(C);o+i  -  Bn+p- 


k=  i 


■  Bn+p 


(5.61) 
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Taking  (5.58),  (5.59),  and  (5.61)  together,  gives  the  following  formula  for  the  diagonals 
{DS  (B)j}f=1: 

DS (C)j,  if  j  <  Jo, 

DS(Bh  =  DS (C)j  +  DS(C) j+i  ~  Hn+p,  if  J  =  Jo,  (5-62) 

DS(C)7+1,  if  J>  JO. 

We  next  show  that  {DSOS);}^  >  {jiN+p>\p~lv  If  l  <  j0  -  1,  then  since  {DSiC)/}^,  > 
we  have 

/  /  / 

Z  ds^j  =  Z  DS(C)i  -  z^«* 

J=1  J=1  «=i 

On  the  other  hand  if  l  >  jo,  we  have 

l  Jo-1 

Z  DS(fi)j  =  Z  DS(C)^'  +  DS(C)i»  +  DS(CWi 

J=i  J=i 

/ 

fJ-N+p  Z  DS(C)7+1,  (5.63) 

«=Jo  +  l 

with  the  understanding  that  a  sum  over  an  empty  set  of  indices  is  zero.  We  continue  (5.63) 
by  using  the  facts  that  {DS(C)7}^j  >  {fiN+p>}pp,=i  and^;V+/+l  >  /uN+p: 

i  i+ 1  /+ 1  i 

Z  DS(fi)j  =  Z  DS^C^'  _  Ahv+/>  -  Z  ~  ^N+p  -  Z  jU,!+a''  (5.64) 

7=1  7=1  «=1  «=1 

Note  that  when  l  =  M,  the  inequalities  in  (5.64)  become  equalities,  giving  the  final  trace 
condition. 

For  the  final  conclusion,  note  that  one  application  of  Chop  Kill  transforms  a  sequence 
{y,„C=  i  with  the  property  that  {DS(C);-}Jj  >  {/iN+p,\ppl=]  into  a  shorter  sequence  {/Jm)"=l 
which  also  has  the  property  that  {DS(5);}^1  >  {fiN+p>)p~'v  As  such,  one  may  indeed  start 
with  AN+P;m  :=  Am  and  apply  Chop  Kill  P  -  1  times  to  produce  a  sequence  of  continued 
outer  eigensteps  that  immediately  satisfies  Definition  24.  □ 
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To  summarize  the  results  of  this  chapter,  note  we  have  completely  solved  the  frame 
completion  problem  given  in  Problem  23:  by  combining  results  of  Theorems  39  and  40, 
we  obtain  the  following  result: 

Corollary  43.  Given  nonnegative  nonincreasing  sequences  {am}^=1,  {fiN+P}p=i  and  {Am}^=l, 
which  satisfies  Am  >  am  for  all  m  =  1  then  {Tm}^=1  is  (a,  /j)-constructible  if  and 

only  if{ DS(A)m}Jf1  >  {/uN+p}pp=l. 
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VI.  Optimal  frame  completions 


In  this  chapter,  we  build  on  the  theory  of  Chapter  5  to  find  the  optimal  ( a,p )- 
constructible  sequence.  The  main  result  of  this  chapter  is  Theorem  48  which  provides 
an  explicit  formula  for  the  optimal  (u,p)-constructiblc  sequence  in  the  special  case  that 
p  =  {/jN+p}p=l  are  all  of  equal  lengths.  In  particular,  we  provide  a  partial  solution  to  the 
following  problem: 

Problem  44.  Given  nonnegative  nonincreasing  sequences  a  =  ! O',,, }')'/= ,  and  p  =  {pN+P}Pp=v 
find  the  (a,  p)-constructible  sequence  which  is  optimal  with  respect  to  the  MSE,  FP,  or  some 
other  given  convex  functional  of  the  eigenvalues  of  the  frame  operator. 

We  are  interested  in  solving  Problem  44  because  of  its  real-world  applications.  For 
example,  if  we  consider  transmitting  an  encoded  signal  over  a  noisy  channel,  it  is  possible 
that  part  of  the  signal  being  transmitted  will  be  distorted  or  even  lost.  As  such,  we  desire 
an  encoding  scheme  that  will  be  as  resilient  to  these  errors  as  possible.  It  is  already  well 
known  that  optimal  frames  in  such  situations  are  UNTFs  as  they  are  minimizers  of  the 
MSE  and  FP  [5,  26].  Constructing  a  UNTF  for  a  given  application  may  not  be  achievable, 
however,  if  given  an  initial  frame,  one  is  restricted  to  adding  only  a  finite  number  of  new 
measurements.  In  such  cases,  the  solution  to  Problem  44  is  not  obvious. 

To  find  the  optimal  ( a ,  //)-constructible  sequence,  we  follow  an  approach  found  in  [40] 
and  [41]  and  measure  optimality  with  respect  to  majorization.  Here  it  is  shown  that  optimal 
frame  completions  are  minimizers  of  a  family  of  convex  functionals  that  includes,  but  is 
not  limited  to,  the  MSE  and  FP.  Later  we  will  show  that  the  optimal  (a, /^-constructible 
sequence  is  the  one  which  is  majorized  by  all  other  sequences  in  the  following  set: 

Definition  45.  Given  nonnegative  nonincreasing  sequences  a  =  {am}^=1  and  p  = 
{pN+p}p=[,  consider  the  set  of  all  possible  (o\//)-constructiblc  sequences  which  according 
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to  Corollary  43  are: 


A{a,n)  :={{Am}n=\  6  adm(or)  :  {Tm}“=1  is  (a,yu)-constructible} 

={{UZ=x  e  adm(cr)  :  {DS(A),}f=1  >  {pN+p}Pp=x}. 

We  begin  in  Section  6.1  with  a  discussion  on  frame  metrics.  In  particular  we  derive 
the  MSE  and  FP  and  then  expand  our  measure  of  optimality  to  include  majorization.  Then 
in  Section  6.2,  we  present  a  partial  solution  to  Problem  44 — an  explicit  formula  for  the 
optimal  (a,/i)-constructible  sequence  in  the  special  case  when  all  [pN+p}pp={  are  of  equal 
lengths. 


6.1  Frame  metrics 

Errors  are  introduced  when  a  signal,  being  transmitted  over  a  communications 
networks,  is  distorted  by  noise  or  when  part  of  it  is  lost  due  to  network  errors.  In 
such  a  situation,  the  original  signal  x  must  be  reconstructed  from  F*x  +  e  where  e  = 
[ei  e2  . . .  eN ]  '  =  Zi)=i  enen,  is  the  zero-mean,  independent,  identically  distributed  added 
noise  with  variance  cr2.  In  order  to  approximate  the  original  signal,  we  apply  the  Moore  - 
Penrose  generalized  inverse  or  pseudoinverse  of  F* .  Therefore,  the  reconstructed  signal 
becomes  (FF*)~1F(F*x  +  e)  which  has  error  given  by  ( FF*)  lFe .  In  order  to  find  the  best 
reconstruction,  we  desire  for  this  error  to  be  a  minimum.  Specifically,  we  integrate  the 
norm  squared  of  the  error  ||(FF*)^1Fe||2  over  M;V  to  derive  the  MSE: 

MSE  =  f  H(FFT 

JRN 


■dl>  n)\l2Pi(£i)P2(e2)  ■ .  .pv(fv) deide2  . .  .dew.  (6.1) 
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Here,  pn(en)  is  the  probability  density  function  for  en.  Writing  p(e)  =  p\(e\) . .  .pN(eN)  in 
(6.1)  gives, 


r  ,  N  N  \ 

MSE  =  l(FF*rlF(  Yj  ^en),  (FFTlF(  ^  enen)\p{e)de 

'  «=1  n=l 


N  N 

ZZ  eneA(.FF*TlFen,(FF*TlFen,)p(e)  de 

n—  1  n'=l 
N  TV 

=  YlYi{{FF*yXFen^FF*rlFen') 

n=  1  «'=1 

Focusing  on  the  integral  of  (6.2),  and  using  the  fact  that  each  en  is  an  independent  and 
identically  distributed  random  variable  with  mean  zero  and  variance  cr2  we  see  that: 


r 

J  RA 


en£„'p(e)  de. 


(6.2) 


r 

J 


r 

J~R 


enen'p(e)de  =  enen’Pi(ei)p2(e2) . . .  pN(eN)de\de2  . .  .dej 


w 


77  =  77 


XOO  ~ 

XI  ](  12  £n'Pn'(€n')den'  I  n  *  n' 


var(6„)  77  =  77' 


0 

Substituting  (6.3)  into  (6.2)  gives: 

N  N 


77  ±  7?' 


7V  7V  y-* 

MSE  =  V  Y{(FF*TXFen,(FF*TlFen, )  enen.p(e) de 

id  id 

N  N 

=  YTj(F{FFr2Fe^e’/y 


n=  1  n'=  1 
N 


e~  77  =  72 

0  77  F  77' 


=  cr2J]<F(FF*)-2Fe„,e„) 

ft=l 

=  cr2Tr(F(FF*)_2F) 

=  cr2Tr((FF*)_1) 

M 


CT‘ 


Zr- 


(6.3) 


(6.4) 


m=l 
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where  {Am}M=1  are  the  eigenvalues  of  FF* .  Since  calculating  the  MSE  involves  inverting 
FF*,  which  can  be  difficult  or  impossible,  another  metric  that  may  be  used  is  the  FP.  The  FP 
is  a  measure  of  the  total  orthogonality  of  the  frame  vectors  [5].  Inspired  by  Columb’s  Faw, 
the  FP  involves  visualizing  the  movement  of  M  charged  particles  restricted  to  M  concentric 
spheres  [9].  The  FP  of  a  sequence  {.fn\N=\  in  MM  is  given  by: 

N  N 

=  (6-5) 

n=  1  n'= 1 

Note  that  (/„,/„')  is  the  in,n') th  entry  of  the  Gram  matrix  F*F,  and  so  the  FP  is  an  L2 
norm  squared  on  the  entries  of  F*F.  Recalling  the  Frobenius  norm  of  a  matrix  A  defined 
by  ||A]||  =  Tr(A*A),  (6.5)  becomes, 

N  N  N  N  M 

FP mNn=x)=Yu  Z  K/'" ^  =Z  Z  ^F*F^n’\2  =  \\F*F\\2f  =  Tr((F F* )2)  =  £  A2n.  (6.6) 

n=  1  n'= 1  n=  1  n'= 1  m=  1 

Minimizing  this  quantity  involves  finding  a  sequence  {fn}^=l  whose  elements  are  as 
orthogonal  to  each  other  as  possible  [35]. 

Note  that  the  MSE  and  FP  are  of  a  similar  form  which  can  be  generalized  to  a  family 
of  convex  functionals.  A  Schur-convex  function  is  one  which  preserves  the  ordering 
of  majorization,  that  is,  <p(x)  >  cpiy)  whenever  x  >  y.  The  following  proposition,  a 
combination  of  results  from  [30]  and  [42],  will  be  useful  in  determining  the  optimal  ( a,/u )- 
constructible  sequence. 

Proposition  46  (Proposition  3.C.I.  of  [38]).  If  I  c  M  is  an  interval  and  g  :  I  — »  M  is 
convex,  then 

M 

fix)  =  g(xm), 

m=  1 

is  Schur-convex  on  1M.  Consequently,  x>y  on  1M  implies  fix)  >  fiy). 

If  g(a:)  =  x2,  then  <p(x)  becomes  the  FP  (6.6),  and  if  g(.r)  =  ^  then  cp(x)  becomes 
the  MSE  (6.4).  The  exponential  function,  fix)  =  Xi»=i  eXm,  is  yet  another  example  of 
a  Schur-convex  function  since  g(a:)  =  ex  is  strictly  convex.  Moreover,  the  converse  of 
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Proposition  46  is  also  true:  if  Yim=i  Sixm)  >  <?(}’m(  holds  for  all  continuous  convex 

functions  g  then  x  >  y  (Proposition  4.B.1  of  [38]).  In  light  of  these  facts,  it  suffices  to  show 
that  the  optimai  (o\//)-constructiblc  sequence  is  the  one  which  is  majorized  by  aii  other 
sequences  in  A(a,ju)  (Definition  45).  Specificaiiy,  we  say  that  a  sequence  {Am}^=l  e  A(a,p ) 
is  the  optimal  (cr,/r)-constructible  sequence  if  {Am}%=l  <  ] Am \^= ,  for  aii  {Am}%=l  e  A(a,p); 
that  is,  if 

k  k 

^  ~A,„  <  'Yj  Am,  'ik  =  1, . . . ,  M  -  1, 

m=  1  m=  1 

M  M 

^  ^  ^  ^  • 

m=  1  m=l 

In  the  next  section,  we  consider  a  speciai  case  when  aii  [pN+p}pp={  are  of  equal  lengths. 
In  this  case,  we  provide  an  explicit  formula  for  the  optimal  (or, //)-constructible  sequence 
and  verify  that  it  is  indeed  optimal  by  showing  that  it  is  majorized  by  all  other  ( a,p )- 
constructible  sequences  in  A(a,p). 

6.2  Construction  of  the  optimal  ( a ,  ^(-constructible  sequence 

In  this  section,  we  construct  the  optimal  (a,  //(-constructible  sequence  in  the  case  when 
all  \pN+p)pp=x  are  of  equal  lengths.  We  begin  by  considering  what  it  means  for  a  sequence 
{Am}^=l  to  be  (cr,yu)-constructible  in  this  special  case,  and  illustrate  how  to  find  the  optimal 
(a,  /r)-constructible  sequence  with  an  example.  The  main  result  of  this  section.  Theorem  48, 
provides  an  explicit  formula  for  the  optimal  (a',/i)-constructible  whenever  {pN+p}Fp=l  are  of 
equal  lengths. 

To  begin,  recall  from  Chapter  5  that  a  sequence  {Am)M=]  is  (^^(-constructible  if  and 
only  if  {DS(A)7}jf;1  >  {pN+P}p=l-  When  {pN+p}pp=x  are  of  equal  lengths,  this  majorization 
requirement  can  be  simplified.  In  this  case,  the  requirement  that  {DS(A)y}^f;1  >  {pN+P}p=l 
comes  for  free  since  any  sequence  of  diagonal  sums  {DS(A)i}^1  majorizes  the  uniform 
sequence  {Pn+p}fp=x  provided  it  has  the  proper  sum  (see  discussion  preceding  Theorem  20). 
As  such,  to  verify  that  {Am}%  is  (a,  //(-constructible,  we  need  only  check  that  it  is 
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nonincreasing  and  that. 


PM  M 

^  l~iN+P  =  -  am)  =  ^  DS(A);.  (6.7) 

p— 1  m=l  7=1 

Moreover,  we  claim  that  when  {/rw+p)p=|  are  of  equal  lengths,  any  (a,/r)-constructible 
sequence  {Am}“=1  must  satisfy: 

crm  <  Tm  <  crm_P  Vm  =  1, . . . ,  M,  (6.8) 

where  am  :=  oo  for  m  <  0.  The  left  hand  side  of  (6.8)  follows  immediately  from  the 
fact  that  any  (ar,//)-constructible  sequence  is  necessarily  a-admissible,  i.e.,  am  <  Am  for  all 
m  =  1, . . . ,  M.  The  right  hand  side  of  (6.8)  follows  from  the  fact  that  subsequent  sequence 
of  continued  outer  eigensteps  generated  by  {Tm}^=1  e  A(a,/u)  must  interlace: 

Am  =  An+P-m  <  An+P- i;m_i  <  d/v+P-2;m-2  <  ’  ’  ’  <  AN+\-,n-(P-l)  <  A^-m-p  = 

It  turns  out  that  if  all  of  the  added  lengths  are  equal,  any  sequence  which  satisfies  both 
(6.7)  and  (6.8)  will  automatically  be  (a,/r)-constructible,  that  is,  {DSIA);}^  >  {hn+p}pp=v 
This  is  not  true  if  {pN+p}p=l  are  of  different  lengths.  For  example,  if  j//,v+i Ttv+2}  =  {\,  § }, 
{a l,a2,a3}  =  {1,2,3},  and  {AUA2,A3}  =  {3,3,3},  Z?n=1(Am  -  am )  =  3  as  required  by  (6.7) 
and  (6.8)  is  satisfied  for  every  Am,  but  {Am}^n=1  is  not  (o',/r)-constructible — specifically, 
§  ^  DS(A)!  =  2.  However,  if  the  same  spectrum  was  built  by  /jN+i  =  /uN+2  = 
{Am}^n=l  will  automatically  be  (a,/r)-constructible  since  |  <  DS(A)j  =  2  and  3  = 
DS(A)i  +DS(A)t  =  2  + 1  =  3.  In  light  of  these  facts,  we  now  turn  to  an  example  illustrating 
how  to  find  to  optimal  (a,/r)-constructible  sequence. 

Example  47.  Let  M  -  3  and  P  -  2.  In  this  example,  we  build  the  optimal  ( a,/u )- 
constructible  sequence  {Am}]n=x  where  a  =  {aua2,a3}  =  {J,  f,  \)  and  /u  =  {^+1,^+2}  = 
{2,2}.  The  initial  sequence  a  is  outlined  in  black  in  Figure  6.1(a).  Just  as  in  previous 
chapters,  we  can  view  the  process  of  building  the  optimal  (a,/i)-constructible  sequence  as 
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iteratively  building  a  staircase.  By  looking  at  the  quantities  we  wish  to  minimize,  we  can 
gain  insight  into  what  the  optimal  staircase  should  look  like.  For  example,  if  we  consider 
the  FP,  since  this  quantity  is  calculated  by  taking  the  sum  of  the  squares  of  the  eigenvalues 
{Am)li=],  we  would  want  to  make  the  highest  steps  in  the  staircase  as  large  as  possible;  that 
is,  we  would  want  to  make  /I3  as  large  as  possible,  followed  by  A2,  and  finally  A\.  Similarly, 
if  we  consider  the  MSE,  we  would  want  to  make  the  lowest  steps  in  the  staircase  as  small  as 
possible.  We  know  that  there  are  limits  on  how  large  each  step  can  be  as  well.  In  particular, 
since  /jn+i  =  /uN+ 2  =  2,  each  Am  must  satisfy  (6.8)  for  all  m  =  1, . . . ,  3,  specifically, 

\  -  ai  <d,  <  cr_  1  =  00, 

f  =  a2  <A2  <  a0  =  00,  (6.9) 

\  =  a3  <'h  <  ari  = 

The  upper  bound  on  Am  is  indicated  by  the  dotted  line  in  Figure  6.1(b). 

To  build  the  optimal  (cr,//)-constructible  sequence,  we  use  a  sort  of  “water- filling'’ 
approach.  Water,  representing  the  total  amount  of  area  that  is  added  to  the  initial  spectrum 
O'-in  this  case  //iV+i  +/uN+ 2  =  4  units  of  area — flows  from  left  to  right  in  the  staircase  picture. 
The  water  flows  at  a  constant  rate  and  will  reach  a  barrier  at  the  dotted  lines  whenever  the 
value  of  Am  is  maximized.  Water-filling  stops  whenever  the  total  volume  of  water  has  be 
used.  The  optimal  spectrum  for  a  and  n  chosen  in  this  example  is  shown  in  Figure  6.1(c). 
The  optimal  (cr,ju)-constructible  sequence  in  this  case  is  {Ai,A2,A2}  =  {y,  21, 1-}. 

We  claim  that  this  water- filling  process  will  always  produce  the  optimal  ( a,/u )- 
constructible  sequence  in  the  case  that  \jiN+p)^=]  are  all  of  equal  lengths.  In  the  following 
theorem,  we  provide  an  explicit  formula  for  this  optimal  water-filled  spectrum  and  prove 
that  it  is  indeed  optimal  by  showing  that  it  is  majorized  by  all  other  sequence  {Am}^={  6 
A(a,n). 
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(a)  (b)  (c) 


Figure  6.1:  Given  existing  spectrum  a  =  {a\,  a2,  a 3}  =  \ }  (a),  we  add  two  additional 

vectors  whose  lengths  are  given  by  {pN+l,pN+2}  =  {2, 2}  so  that  the  resulting  spectrum  is  the 
optimal  (a,  //)-constructible  sequence.  Since  pN+ 1  =  pN+2  =  2,  by  (6.8),  am  <  Am  <  2 

for  all  m  =  1, 2, 3.  The  upper  bound  on  Am  is  indicated  by  the  dotted  line  in  (b).  Since  no 
dotted  line  is  present  at  levels  m  =  1  and  m  =  2,  the  upper  bound  is  infinite.  Water-filling 
with  pN+i  +  pN+2  =  4  units  of  volume  yields  the  optimal  (o\//)-constructiblc  sequence 
{Ai,A2,  T3}  =  { y,  y,  |}  indicated  by  the  gray  area  in  (c). 


Theorem  48.  Let  {am}ffl=l  and  j^,y+/d^=1  be  nonnegative  nonincreasing  sequences  and 
suppose  p  =  j  Yfp=  1  Pn+p ■  For  m  =  1, . . . ,  M,  define, 

Vm  :=  (am- 1  -  am)  min{M  -  m  +  1,  P],  (6.10) 

M  M 

Wm  -=J]vk  =  ~  crk)min{M  -k+  l,/5},  (6.11) 

k—m  k=m 

where  at)  :=  00.  Define  {Am}ffj=l  according  to  the  following  rule:  Pick  any  j  such  that 
Wj+ 1  <  Pp  <  Wj,  define: 


1  <  m  <  j  —  1 , 

aJ  +  min{  M-j+ 1  ,P)  yP  ^j+ 1  j  > 

j  <  m  <  j  +  min{M  -  j  +  1,  P}  -  1, 

(6.12) 

&m-Pi 

j  +  min{M  -  j  +  1,  P}  <  m  <  M. 

Then  {Am}^=l  6  A(a,p)  and  {Am}ffl=]  is  the  optimal  (a,  p)- co nst ru c t i h l e  sequence,  that  is, 
UmC=1  <  {Am}^=x  for  all  {Am}M=]  e  A(a,p). 

Proof  To  prove  that  (6.12)  is  optimal,  we  must  show  that  it  is  (o',/i)-constructible  and  that 
it  is  majorized  by  every  sequence  {Tm}^=1  in  A(a,p).  We  begin  by  showing  that  (6.12)  is 
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(a,yu)-constructible,  i.e.,  {DS(A);}^1  >  {pN+P}p=l-  By  assumption  {nN+p}p=l  are  all  of  equal 
lengths,  so  in  order  for  {DS(A)i}^1  >  {nN+p}p=v  it  suffices  to  show  that  X f=  i  DS(A)V  =  Pjj. 
To  do  this,  we  first  show  that  Xm=i  Xn  =  Zilti  4m.  Summing  (6.12)  for  all  M: 


To  do  this,  we  first  show  that  YXL\  X 

M  j- 1  y+min{M-y+l,P}-l 


m  —  Zjm=l 


m=l  m=l 

./■-i 


+  ^  Q'm-P 

m— j+min{M-j+l,P} 


^  ^  ^ m-P 
■j+min{M-j+l,P} 


(6.13) 


Z  =  Z  °'m  +  Z  P  +  min \M-  /  +  “  W/+lj  +  Z 

m=l  m=l  m=j  J  ’  m=y+min{M-7+l,/>} 

7-1  M 

=  Z®'"  +  min{A/  -  j  +  l,P}aj  +  Pju  -  Wj+1  +  (6.13) 

m=  1  m— 7+min{  M-j+ 1  ,P} 

Substituting  the  fact  that  Pjj  =  Tjp=iPn+p  =  Zm=i(4  ~  «,„)  for  any  arbitrary  sequence 
Um}^=1  6  A(a,/j),  (6.13)  becomes: 

M  M  j- 1  M 

^  ^  'h/  ^  ^(d-m  ®m)  "t”  ^  ^  ^  ^  &m-P  IU i U { A/  j  4  1,  F }  O  j  —  W)+j 

m=l  m=l  m=l  »i=_/+min{M-y+l,P} 

MM  M 

=  ^  d,„  -  ^  cr„,  +  ^  «m-p  +  min{M  -  j  +  1 ,  P}a7-  -  1T/+i.  (6. 14) 


iKi  iKi  J  A  IV  1 

^  ^  4/h  ^  'jAm  ®m)  "I"  ^  ^  ^  ^  Q-m—P  "t"  m i n { A/  j  +  1,  P\(X j  —  Wj+\ 

m=  1  m=l  m=l  Hi=7+min{M-y+l,P} 

MM  M 

=  ^  T,„  -  ^  cr,„  +  ^  «,„_/>  +  min{M  -  j  +  1  ,P}aj  -  Wj+].  (6.14) 

m=  1  m—j  m— j+minj  M-j+ 1  ,P] 

To  further  simplify  (6.14),  we  consider  two  different  cases.  First,  we  consider  the  case 
when  min{M  -  j  +  1,  P}  =  M  -  j  +  1.  In  this  case,  (6.14)  can  be  simplified  as  follows: 

M  M  M 

Z  ~*m  =  Z  A,n  -  Z  a,n  +  -  j  +  1  ^ai  -  WP >•  (6.15) 

m=\  m=  1  m—j 

Next,  we  substitute  the  definition  of  Wj+ 1  from  (6.11),  at  which  point  (6.15)  becomes: 

M  M  M  M 

'j  ^  Am  —  ^  ,  Am  'j  ^  <xm  +  (M  —  j  +  1 ) o  /  'j  ^  ( y,n ) ( A1  m  +1) 

m=  1  m=  1  m— j  m— j+ 1 

MM  M 

—  ^  ^  Am  ^  ^  (-^  j  1)^7  (-^  j  ^  j 

m=  1  m= j  m— j+ 1 

M 


(6.14) 


7i=  1  m— j 


and  so  Xm=i  Xn  =  Xm=i  in  the  case  that  min{M  -  j  +  1,  P)  =  M  -  j  +  1,  as  claimed.  Next, 
we  consider  the  case  when  min{M  -  j  +  1,  P)  =  P.  In  this  case,  (6.14)  becomes 


M  M  M  M 

'j  ^  Am  'j  ^  Am  —  'j  ^  am  +  'j  ^  (}>n- /’  +  Poij  1 1  i  • 

m=  1  m=  1  m= j  m—  j+P 


(6.16) 
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Again,  substituting  (6.11)  for  Wj+ 1,  (6.16)  becomes: 

M  M  M  M 

^  Xn  =  ^  d rn  -  a,n  +  Pa  j  ~  ^  (am- 1  -  «m)  min{M  -  m  +  1,  P)  (6.17) 

m=l  m=  1  m=M-P+ 1  m= 7+ 1 

We  can  further  simplify  (6.17)  by  splitting  the  last  summation  into  two  depending  on 
whether  min [M  -  m  +  1  ,P}  =  M-  m  +  1  or  min{A7  -  m  +  1 ,  P)  =  P: 

M  M  M  M-P  M 

^  ' F ;  —  ' F ;  ^  ^  !  P(Xj  ^  ^  (®m-l  aln)P  ^  Crm)(ilf  /7Z+1).  (6.18) 

m=l  m=l  m=M—P+  1  m= 7+ 1  m=M-P+l 

Notice  in  (6.18),  that  two  of  the  four  sums  on  the  right  hand  side  of  the  equation  are 
telescoping  sums  and  so  simplifying  (6.18)  results  in  many  terms  that  cancel  each  other 
out  yielding: 

M  M  M  MM 

^  'j  A /;  —  ^  ' j  dm  ^  '  1  a,n  +  PO! j  P&j  Po^M-P  Pa  M-P  ^  .  am  ~  ^  ^  d m ■ 

m=  1  m=l  m=M-P+l  m=M-P+l  m=l 

Hence,  2m=i  dm  =  Z,!f=i  dm  in  the  case  that  min{A7  -  j  +  l,  P)  =  P.  Having  this  result,  we 
are  now  able  to  prove  that  (6.12)  is  (a,/i)-constructible.  Since  {nN+p}pp={  are  all  of  equal 
lengths  by  assumption,  in  order  for  (6.12)  to  be  (o',/i)-constructible,  it  suffices  to  show  that 
ZjiDS(A  )j  =  Pfi: 

MM  M  M  P 

^  ^  DS(A) j  —  ^  'ddm  a]n)  —  ^  ^(dm  crm)  —  ^  ^  DS(A)y  —  ^  n^+p  —  Pfu, 

j=  1  m=l  m=l  J  1  /)=  1 

so  indeed,  (6.12)  is  (a,/7)-constructible. 

Finally,  we  claim  that  (6.12)  is  optimal.  In  order  to  prove  this,  we  must  show  (6.12)  is 
majorized  by  all  other  (cr,yu)-constructible  sequences  in  d(a,ju).  That  is, 

k  k 

^  Am  >  £  d„„  Vk  =  1, . . . ,  M  —  1,  (6.19) 

m—  1  m- 1 

M  M 

X  A,n  =  X  /*m’  (6.20) 

m=l  m=  1 

for  all  {dm}))f=1  6  A(a,/j).  We  just  finished  proving  that  (6.20)  holds  in  order  for  (6.12)  to  be 
(cr,/r)-constructible,  and  so  we  are  left  to  show  that  (6.19)  holds  for  k  =  l, . . . ,  M  -  For 
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1  <  k  <  j  -  1,  (6.19)  follows  immediately  from  (6.12)  and  the  left  hand  side  of  (6.8): 

k  k  k 


^  Am  ^  ^  ocm  <  ^  i  Am. 


m-\  m=  1  m=l 

Similarly  for  j  +  min {M  —  j  +  l,P)  <  k  <  M,  (6.19)  follows  from  (6.12)  and  negating  the 
right  hand  side  of  (6.8);  that  is,  -am-P  <  -Am  for  all  nr. 

k  M  M  M  M  M  M  k 

^  ^  Am  —  ^  Am  ^  ,  An j  —  ^  ^  Am  ^  ^  O^m-P  —  ^  .  Am  ^  Am  —  ^  ^  Am. 

m=  1  m—  1  m=£+l  m=l  m=k+ 1  m=l  m=k+ 1  m=l 

For  the  remaining  case  when  j  <  k  <  min{M  -  j  +  1,  P}  -  1,  we  first  claim  that 

M  k 


min{M-j+  1  P 1(P/"  “  ^y+l)  Z  (/lm  “  a"l)  +  Z  /lf"  _  “  2)°w  (6.21) 


(&  -  ./  +  1) 

— tr//  -  wy+i;  s  r/u  - 

m=j+ 1  m=7+l 

To  show  that  (6.21)  is  in  fact  true,  we  consider  two  separate  cases — when  At  <  Ak  and 
\  >  Ak.  For  the  case  when  \  <  Ak,  first  note  for  any  j  <  k  <  minjM  —  j  +  1,  P)  -  1,  (6.12) 
gives 

(k  -  j  +  1) 

“pj W  ~  VVJ+D  = 

‘  m=j 


i  \  K 

j+\  P](pp - Z ~Am -(k~j+ ])ai- 


(6.22) 


min{M  -  j 

In  order  to  simplify  (6.22),  note  from  (6.12)  that  all  values  of  Am  are  equal  whenever 
j  <  m  <  min {M  -  j  +  1,  P)  -  1.  Combining  this  with  the  case  that  \  <  Ak,  we  have 

Aj  =  Aj+i  =  ■  ■  ■  =  Ak  <  Ak  <  Ak- 1  <  •  •  •  <  Aj, 

at  which  point  (6.22)  becomes: 

-  i *-■ - » -  > + ^ 

J  m= j 


=  (Aj  -  aj)  +  ^  Am  ~(k-  j)aj 
j=j+ 1 

j  k 

<  Z(/i/ _  ai> +  Z Am  ~ (k  -  ',)ai 

m=  1  j=j+ 1 

M  k 

—  Pfi  ^  ^  (dm  ®m)  “t-  ^  ^  (k  ~  j)a j 


m= 7+1 


m= 7+ 1 
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Thus,  (6.21)  holds  in  the  case  that  Ak  <  Ak. 

In  the  second  case  when  Ak  >  Ak,  first  note  that  since  (6.12)  is  (cr,//)-constructible  and 
Am  =  am  for  m  =  1 -  1 : 

M  7-1  M  M 

^  ^(4»|  ®m)  —  ^  ^(crm  0!m)  +  j  (  ■  i  n:  (Y  ni )  j  J  Aff,  (},/;  ).  (6.23) 

m=  1  m=  1  m=7  m=j 

Substituting  (6.23)  into  the  right  hand  side  of  (6.21)  gives 

MM  M  MM 

Pfi-  ^  4,,,+  ^  am-(k-j)aj  =  ^(4m-arm)-  ^  4m  +  ^  am-(k-j)aj.  (6.24) 

ra=£+l  m=7+l  m-j  m=k+ 1  m=j+ 1 

Next,  we  use  our  assumption  that  /t/(  >  Ak  and  the  fact  that  all  values  of  Am  are  equal 
whenever  j  <  m  <  min {M  -  j  +  1,P}  -  1  ;  specifically,  this  implies 

^■j+mm{M-j+\,P}-\  —  •  •  •  —  Ak+[  —  Ak  >  Ak  >  Ak+  j  •  •  •  >  /fj+minj^f_y+i  p|_i , 


and  so 


min{A/-;'+l,P}-l  minIM-j+l,/5}-! 


m=k+ 1 


m=k+ 1 


(6.25) 


Combining  the  right  hand  side  of  (6.8)  with  the  fact  that  Am  =  crm_p  for  m  =  m i n { M  -  j  + 
1,  P}, . . . ,  M,  we  obtain  a  similar  expression: 

MM  M 


'j  ^  'C? 

m=min{  M-j+ 1  ,P} 


^  (Ym-i‘  4  ^  4„ 

m=min{M-y+l,P}  m=min{M-7+l,P} 


(6.26) 


Having  (6.25)  and  (6.26),  we  continue  (6.24): 

MM  M  MM 

PfA  'j  ^  4,,;  +  'j  ^  O  ' (^  ./  )*  1/  —  ^  /4m  crm)  ^  ^  4m  +  ^  ^  crm  ( k  j)( i  . 

m=£+l  m=7+l  m-j  m=k+ 1  ra= y+1 

k  M  M 

^  ^  ^  ^  "I-  ^  ^  j)^  j 

m— j  m— j  m= j+ 1 

k 

=  J]ln-(k-j+  l)aj.  (6.27) 

m=j 
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Finally,  we  substitute  for  Am  according  to  (6.12)  in  order  to  obtain  (6.21): 


M  M 

pp-  Y  Am  +  Y  a,n  ~ {k  ~  ^a> 

m=k+ 1  m=j+ 1 


>(k-j+  1) 


<*j  + 


1 


(k  -  j  +  1) 


min{M  -  j  +  1 ,  P) 


r( P~P  ~  Wj+l). 


(Pp  -  Wj+ 1) 


(k  ~  j  +  1  )(V  j 


min{M  -  /  +  1 ,  P) 

Having  that  (6.21)  is  indeed  true,  we  now  show  (6.19)  in  the  case  that  j  <  k  <  min{M  —  j  + 
hP}-h 

u  i  n 

1 


k  7-1  k 

^  \  'kn  ^  ^  OSm  "t"  ^  ,  OC  j  + 


m=  1  m=  1 

7-1 


m=7 


^  o',,,  +  (k-  j+  \)aj  + 


min{M  -  j  +  1 ,  P) 
(k~  j+  1) 


m=  1 
7 


min{M  -  j  +  1 ,  P) 


( P~P  -  WJ+  Oj 

(^  -  Wj+l) 


M 


—  ^  \  &m l  1  Pp  ^  ^  (/  hi  ®m)  “1“  ^  ^  '  h 


m=l 

j 


m— j+ 1 


m= 7+ 1 


^  ^  "I”  ^  ^ (^m  ^m)  ^  ^ 


m=l  m=l 

j 


m= j+ 1 


^  ^  Am. 


m—  1 


Therefore,  {Tm}^=1  is  majorized  by  every  sequence  {dm}^=1  e  and  so  it  is  the  optimal 

(of,//)-constructible  sequence.  □ 


We  leave  the  construction  of  the  optimal  ( a ,  /i)-constructible  sequence  for  any  arbitrary 
set  of  lengths  {jiN+p\pp=]  for  future  work. 
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VII.  Conclusions  and  Future  Work 


In  this  dissertation,  we  considered  the  problem  of  constructing  every  frame  whose 
frame  operator  has  a  given  spectrum  and  whose  vectors  have  prescribed  lengths.  Regardless 
of  building  a  frame  from  scratch  or  completing  a  frame  from  preexisting  measurements, 
the  solution  to  this  problem  involved  a  two-step  process — first,  picking  a  sequence  of 
eigensteps  and  second,  constructing  the  frame  vectors  one  by  one.  We  also  considered 
the  problem  of  finding  the  optimal  frame  completion,  and  were  able  to  prove  optimality 
in  the  case  where  the  set  of  lengths  are  all  equal.  While  we  feel  confident  that  we  know 
intuitively  what  the  solution  should  be  in  general,  proving  optimality  in  all  cases  has  proved 
to  be  difficult. 

This  leads  directly  into  our  future  work.  When  we  consider  optimally  completing  a 
frame  for  any  arbitrary  set  of  lengths,  any  solution  we  have  proposed  to  date  has  been  in  the 
form  of  an  algorithm.  The  problem  with  this  method  is  that  it  does  not  lend  itself  well  to  the 
proof  techniques  that  we  currently  use  or  to  prove  that  the  conjectured  optimal  sequence 
is  indeed  optimal.  The  authors  of  [40]  have  already  proposed  their  own  algorithms  for 
solving  for  the  optimal  frame  completion  problem,  but  their  results,  like  our  own,  are  just 
conjectures  at  this  point.  As  such,  our  future  work  revolves  around  formally  proving  the 
solution  we  propose.  We  do  not  include  our  solution  algorithm  for  the  optimal  ( a,/u )- 
constructible  sequence  here,  rather  we  close  with  an  example  illustrating  how  it  works. 

Example  49.  Our  goal  is  to  construct  the  optimal  (a,  //)-constructible  sequence  for  a  frame 
obtained  by  adding  P  =  4  additional  vectors  to  a  set  of  preexisting  vectors  in  M3.  For  this 
example,  a  =  \a\,  ar2,  <*3}  =  1 1,  |,  |],  and  the  added  vector  lengths  //  =  {/aN+p}4p=l  are  given 
by  {/jn+i,iun+2,ij,n+3}  =  {2, 1,  |,  — the  same  values  used  in  Example  42.  We  know  from 

Corollary  43,  that  a  sequence  is  (cr, //)-constructible  provided  its  diagonal  sums  majorizes 
the  set  of  added  lengths.  This  majorization  requirement  involves  taking  partial  sums  of 
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diagonal  sums  as  defined  in  Definition  37.  As  such,  we  give  a  special  name  to  these  partial 
sums: 

Definition  50.  Let  A  €  Define  the  triangular  sum  function  TS  :  MM  — »  M;V/ 

M 

[TS(A)],-:=£DS(A)*.  (7.1) 

k=j 

For  this  example,  we  can  use  Definition  50,  to  reformulate  the  original  majorization 
inequalities  given  by  (5.24)  and  (5.25)  in  terms  of  these  new  triangle  sums.  Specifically, 
in  order  for  {Tm}^=1  to  be  (or,  //)-constructible,  i.e.,  {DS(A)j}3j=i  >  {/uN+p}p=v  the  optimal 
spectrum  we  will  build  must  satisfy  the  following: 

4  3 

DS(A  )j  =  TS(A)m, 

p'—m  j—m 

4  3 

Z^'  =  z  DS(A)y-  =  TS(A)i,  (7.2) 

p'= i  j=  i 

for  m  =  1, . . . ,  3.  We  refer  to  (7.2)  as  the  triangle  sum  constraints  throughout  the  rest  of 
this  example. 

Again,  we  can  visualize  the  process  of  constructing  the  optimal  (ar^-constructible 
sequence  by  building  a  staircase  just  as  we  did  in  Examples  15  and  42,  with  a  being 
the  initial  set  of  steps.  To  build  {/tm)~’j=1,  we  use  the  same  “water- filling”  approach  used 
in  Example  47.  Water,  representing  the  total  amount  of  area  that  is  added  to  the  initial 
spectrum  a,  flows  from  left  to  right  in  the  staircase  picture.  The  water  flows  at  a  constant 
rate  and  will  reach  a  barrier  if  equality  occurs  in  one  of  the  triangle  sum  constraints.  For 
our  example,  if  TS(A)2  =  £p=2  Hn+p,  TS(A)2  is  “full,”  meaning  water  is  barred  from 
the  second  and  third  lower  diagonals.  A  barrier  is  placed  to  prevent  water  from  flowing 
into  these  diagonals,  and  water  is  restricted  to  filling  in  the  first  diagonal,  DS(A)i,  from 
that  point  forward.  Water  filling  stops  when  the  total  area  requirement  has  been  met, 
that  is  Yjm=Mm  -  a>n)  =  IjPp=i  l*N+P — particularly  for  this  example,  Xm=i(T«  -  a,n)  = 
2p=i  Hn+p  =  3-  In  contrast  to  finding  the  optimal  (a,  yu)-constructible  sequence  when  all 
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the  added  lengths  are  equal,  we  are  unable  to  determine  where  the  barriers  (dotted  lines  in 
Figure  6.1(b))  should  be  placed  a  priori. 

We  now  simulate  this  water-filling  process  by  introducing  variables  into  the  optimal 
spectral  partition  matrix  we  wish  to  build.  We  begin  with  the  spectral  partition  matrix  for 
a  and  let  A3i3  =  x. 

0  1  i  I 

u  1  4  2 

A=  0  0  i  |  (7-3) 

0  0  x  | 

In  order  for  (7.3)  to  be  constructible,  X/=m  DS  (  A)  j  <  X)Lm/W+p  for  m  =  1,...,3,  and 
(7.3)  must  also  be  cr- admissible  according  to  Definition  35.  Specifically,  Amn  >  0  for 
all  m  =  1, . . . ,  3  and  n  =  1, . . .  ,4,  and  we  much  check  (ii)  that  Amn  <  Am_i;„  for  all 
m  =  2, . . . ,  3  and  n  =  1, . . . ,  4.  In  light  of  these  facts,  x  must  be  chosen  so  that  the  following 
inequalities  occur  simultaneously: 

TS(A)j  =x<l 

TS(A)2  =  0  <  §  1 

x  — 

TS(A)3  =  0  <  \  4 

o<*<  \ 

Substituting  x  =  |  into  (7.3),  and  tracking  the  area  in  each  triangle  sum,  we  have: 

0  1  i  5  TS(A)1  =  i<2 

A=  0  0  i  |  TS(A)2  =  0  <  |  (7-4) 

0  0  i  i  TS(A)3  =  0<| 

At  this  point,  summing  across  the  rows  of  A  gives  that  A\  =  |,  d2  =  |,  and  d3  =  |. 

Since  Xm=i(dm  -  am)  =  \  the  total  area  requirement  has  not  been  met.  Also  note 
that  since  none  of  the  triangle  inequalities  in  (7.4)  have  been  saturated  (i.e.,  no  equalities), 
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no  barriers  have  been  placed  and  water  filling  continues  from  left  to  right  in  the  staircase 
picture.  Continuing  this  process,  we  now  let  A2,2  =  A3-2  =  x. 


0  1  i 

0  X  \ 


2 

_j_ 

2 


0  x  l 


1 

2 


Again  we  require  the  resulting  A  to  be  constructible,  which  amounts  to  finding  the  value  of 
x  such  that  the  following  inequalities  occur  simultaneously: 


TS(A)1=2.v+i<| 

TS(A)2  =x<\ 

<  =>  x  =  1 

TS(A)3  =  0  <  \ 

0  <  x  <\ 


TS(A)j 


2  <  1 
4-2 


A  = 


0 


1 

2 


TS(A)2  =  1  <  I 


0  1 


TS(A)3  =  0  <  \ 


(7.5) 


Summing  across  the  rows  of  A  gives  that  di  =  d2  =  d3  =  |.  The  total  amount  of  area  that 
has  been  added  at  this  point  is  X,«=i(T,n  -  am)  =  \  <  7,  which  gives  |  units  of  area  left 
to  water-fill  with.  Again,  none  of  the  triangle  inequalities  in  (7.5)  have  been  saturated,  so 
water  filling  continues  from  left  to  right.  Next,  we  let  Ay  =  A2>1  =  A3>1  =  .v. 


A  = 


1, 

2 

1 

2 


1 

2 
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The  following  inequalities  must  be  satisfied  in  order  for  A  to  be  constructible: 


TS(A)i  =3x+l  <  \ 

TS(A)2  =  2x  +  1  <  | 

TS(A)3  =  *  <  \ 

0  <  X 

Here  we  see  that  the  value  of  x  =  ^  is  a  result  of  TS(A)2  reaching  its  maximum  value  of 

\  1  S  \  TS(A)1=3  <1 

I  1  i  i  TS(A)2  =  §  <  I  (7.6) 

i  l  i  i  TS(A)3  =  i<i 

Barriers  are  placed  in  (7.6),  indicated  by  the  shaded  bands  in  A,  to  prevent  water  from 
flowing  into  diagonals  two  and  three.  The  remaining  | units  of  area  left  to  water  fill  with  is 
restricted  to  filling  in  the  first  row  of  A.  As  such,  we  let  x  be  the  amount  of  area  that  can  be 
added  to  Au. 


Choosing  x  so  that  the  following  inequalities  are  satisfied  simultaneously, 

TS(A)j  =  jc  +  3  <  \ 

TS(A)2  =  §<§  1 

TS(A)3  =  \<\  1 

0<x 

gives  x  =  \  which  is  a  result  of  the  total  sum  requirement  being  met,  i.e.,  TS(A)i  has 
reached  its  maximum  value.  The  water  filling  is  now  complete  and  the  resulting  optimal 
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(cr,/r)-constructible  matrix  has  been  found. 


TS(A)i  =  \  <  \ 

TS(A)2  =  I  <  I  (7.7) 

TS(A)3  =  |  <  \ 

Notice  that  all  of  the  diagonals  of  (7.7)  are  now  shaded  because  TS(A)i  has  been  saturated. 
That  is,  XLi(4  _  am)  -  2p=i  ri/V+p-  Summing  across  the  rows  of  (7.7)  gives  the  optimal 
(cr,/r)-constructible  sequence  to  be: 


3 

1 

1 

1 

4 

4 

2 

A  = 

1 

1 

1 

1 

4 

4 

2 

1 

1 

1 

1 

4 

4 

2 

=  2. 


We  claim  that  this  is  the  sequences  that  is  majorized  by  all  other  sequences  which  can  be 
built  from  a  and  //.  For  example,  if  we  compare  this  optimal  (cr,yu)-constructible  sequence 
to  a  nonoptimal  one  such  as  the  one  given  in  Example  42,  we  see  that 


13  ~  ~  ~  13 

—  -  A\  +  T2  +  T3  —  A\  +  A2  +  T3  -  — , 


11 

T 


—  +  T2  >  T]  +  A2 


9 

r 


13 

—  =  Ai  >  A\ 


5 

2’ 


and  so  {^,  1}  >  {|,2, 2}.  We  leave  the  proof  that  {Tm}^=1  is  majorized  by  all  {/lm|^=|  6 

A(a,  n)  for  future  work. 


142 


Appendix:  MATLAB  code 


The  following  MATLAB  code  implements  the  algorithm  given  in  Theorem  7.  For  the 
sake  of  simplicity,  Vn  =  I  for  all  n.  The  following  two  functions  must  be  placed  in  the  same 
directory  in  order  to  execute  the  code. 

The  first  function  constructU.m  implements  Step  B  of  Theorem  7.  Here  the 
output  of  the  function  is  a  slightly  modified  version  of  Steps  B.4  and  B.5.  The  function 
constructU.m  returns  U*fn+l  and  U*U  v  We  recursively  call  constructU.m  and  then 
multiply  the  outputs  at  each  iteration  in  order  to  calculate  fn  and  Un  for  n  =  2, . . . ,  N .  This 
step  is  accomplished  by  the  function  constructFrame  .m  which  outputs  the  final  sequence 
of  vectors  F  =  {f„}„=l  whose  frame  operator  FF*  has  spectrum  and  which  satisfies 

WfnW2  =  Fn  for  all  n. 

function  [U,  Uf]  =  constructU(El ,  E2) 

%  Description:  This  function  implements  Steps  B.l-5  of  the  algorithm  to 
%  explicitly  construct  any  and  all  sequence  of  vectors  whose 

%  partial-frame  operator  spectra  match  the  eigensteps  chosen 

%  in  Step  A.  Here,  we  assume  VI,..., Vn  are  the  identity. 


%  Call: 

[U,  Uf]  =  constructU(El ,  E2) 

% 

El  =  Spectrum  at  (n) 

% 

E2  =  Spectrum  at  (n+1) 

%  Output: 

U  =  U_ (n) *  U_(n+1) 

% 

Uf  -  U_ (n) *  f_(n+l) 

%  File: 

constructU.m 

%spectra  must  be  row  vectors  and  listed  in  decending  order” 
if  ~isrow(El),  E1=E1’;  end 
if  ~isrow(E2),  E2=E2’;  end 
El  =  sort(El,  ’descend’); 

E2  =  sort(E2,  ’descend’); 

M  =  length(El) ; 


%Find  indices  of  unique  elements  (Step  B.  2)***************' 
R1  =  El;  %Unique  set  of  eigenvalues  of  El 
R2  =  E2 ;  %Unique  set  of  eigenvalues  of  E2 
for  i  =  1:M 

[tf,  loc]  =  ismember(El (i) ,  R2) ; 
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if  tf  ==  1 

[tf , loclast]=ismember(El(i) ,  Rl) ; 
Rl(loclast)  =  -1; 

R2(loc)=-l ; 

end 

end 


%Index  sets  of  unique  elements  of  El  and  E2,  respectively. 
I  =  find(Rl  >=Q) ; 

J  =  find(R2  >=Q) ; 

Ml  =  length(I) ; 

M2  =  M-Ml; 


Rl  =  Rl (I) ; 
R2  =  R2(J); 


%Construct  column  and  row  vectors  (Step  b. B) ********************* 
for  i  =  1 : Ml 


P(i)  =  sqrt(-prod(Rl(i)-R2)/prod(Rl(i)-Rl(find(Rl~=Rl(i)))))  ; 
Q(i)  =  sqrt(prod(R2(i)-Rl)/prod(R2(i)-R2(find(R2~=R2(i))))) ; 


%Construct  difference  matrix 
for  i  =  1 : Ml 

D(i , :)  =  1 . / (R2-R1 (i)) ; 

end 

W  =(P’*Q).*D; 


%Create  Block  Diagonal  Matrix 
UU=blkdiag(W, eye (M2)) ; 

%Permute  Row  and  Columns ; 

PRow  =  permMat(I ,M) ; 

PCol  =  permMat(J ,M) ; 


%Compute  U  and  Uf  (Modified  Steps  B.4  and  B.5)' 
U  =  PRow*UU*inv(PCol) ; 

Uf=zeros(M , 1) ; 

Uf(I)  =  P; 

% . 


%Define  permutation  matrix  given  the  unique  index  set  I 
function  P  =  permMat(I,M) 
m  =  1 :  M ; 

pi  =  [I  setdiff(m, I)] ; 

Id  =  eye(M) ; 
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P  =  Id(l :M,pi) ; 


function  tf  =  isrow(E) 
[rows , cols]=size(E) ; 
if  rows==l 
tf=l; 

else 

tf=® ; 

end 


function  F  =  constructFrame(E,Ul) 

%  Description:  This  function  implements  Step  B  of  the  algorithm  to 

% 

% 

% 

%  Call: 

% 

% 

%  Output: 

%  File: 


explicitly  construct  any  and  all  sequence  of  vectors  whose 
partial-frame  operator  spectra  match  the  eigensteps  chosen 
in  Step  A.  Here,  we  assume  VI,..., Vn  are  the  identity. 

=  constructFrame(E ,  Ul) 

E  =  Matrix  of  eigensteps 
Ul  =  Initial  unitary  matrix 
The  frame,  F. 
constructFrame . m 


UU(: , : ,1)  =  Ul; 

U(: , : ,1)  =  UU(:,:,1); 
F(: , 1)  =  UC: ,1,1); 

[M , N]  =  size(E) ; 


for  i  =  2:N 

[UU( : , : , i) ,  Uf(:,i)]  =  constructU(E( : , i-1) , E( : , i)) ; 
U( : , : , i)  =  eye(M) ; 

%Multiply  matrices  to  find  new  U 

for  j  =  1 : i ,  U(:,:,i)  =  U( : , : , i)*UU( : , : , j) ; end 

%Multiply  to  find  new  f 
Temp  =  eye(M); 

for  j  =  1 : i— 1 ;  Temp  =  Temp*UU( : , : , j) ;  end 
F( : , i)  =  Temp*Uf ( : ,i) ; 


The  following  example  reproduces  the  results  of  Example  8  where  the  eigensteps  are 
given  by  (3.28).  For  the  sake  of  simplicity,  U\  =  I  and  V„  =  I  for  n  =  1, . . . ,  4. 

»  E  =  [00®  2/3  5/3 ;Q  1/3  4/3  5/3  5/3 ; 1  5/3  5/3  5/3  5/3] 
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000  0.6667  1.6667 

0  0 . 3333  l.BBBB  1.6667  1.6667 

1.0000  1.6667  1.6667  1.6667  1.6667 

»  Ul=eye(3) 

U1  = 

1  0  0 

0  1  0 

0  0  1 

»  F  =  constructFrame(E,Ul) 


1.0000  0.6667  -0.4082  -0.1667  0.1667 

0  0.7454  0.9129  0.3727  -0.3727 

000  0.9129  0.9129 


The  following  MATLAB  code  constructs  a  sequence  of  continued  outer  eigensteps 
using  the  Chop  Kill  algorithm  given  in  Theorem  40: 
function  eigensteps=CK(lambda, alpha, mu) 

%  Description:  This  function  implements  the  Chop  Kill  (CK) 

%  Algorithm  and  creates  a  sequence  of  eigensteps  for  lambda. 

%  Call:  eigensteps  =  CKflambda, alpha, mu) 

%  lambda  =  final  spectrum 

%  alpha  =  initial  spectrum 

%  mu  =  set  of  lengths 

%  Output:  table  of  eigensteps 

%  File:  GTK.m 


%spectra  must  be  row  vectors  and  listed  in  decending  order***************** 
M  =  length(alpha) ; 

P  =  length(mu) ; 

T(:,:,P)  =  table (lambda, alpha) ; 
eigensteps  =  zeros(M,P+l) ; 

eigensteps( : ,P+1)  =  flipud(sum(table (lambda, alpha) , 2)) ; 
eigensteps( : , 1)  =  flipud(sum(table(alpha , alpha) , 2)) ; 
for  p  =  P: -1:2 

newT  =  T ( : , : , p) ; 

newT (f ind(tril (newT) ) )=© ; 
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nu(M, l)=mu(p) ; 
for  j  =  M : -1 : 1 

for  k  =  M-j+1 : -1 : 1 
if  k==(M- j+1) 

delta  =  min(nu(j-l+k,10  ,  T(j-l+k,k,p)) ; 

else 

delta  =  min(nu(j-l+k,10  ,  T(j-l+k,k,p)-T(j+k,k,p)) ; 

end 

newT(j-l+k,10  =  T(j-l+k,k,p) -delta; 
if  k>l 

nu(j-2+k,k-l)  =  nu(j-l+k,k)  -  delta; 

else 

nuCM,M-j+2)  =  nu(j-l+k,k)  -  delta; 

end 


end 

end 

eigensteps( : ,p)  =  flipud(sumCnewT, 2)) ; 

T( : , : ,p-l)  =  newT; 

end 
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